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^  G  A  O 

^^^Jj^^^Accountability  *  Integrity  *  Reliability _ 

United  States  General  Accounting  Office 
Washington,  D.C.  20548 


December  29,  2000 

The  Honorable  Janet  Reno 
The  Attorney  General 

Dear  Madam  Attorney  General: 

This  report  addresses  the  Immigration  and  Naturalization  Service's  (INS) 
management  of  information  technology  (IT)  investments.  Each  year  INS 
invests  hundreds  of  millions  of  dollars  on  IT  systems  and  activities.  We 
formd  that  INS  has  established  some  important  capabilities  for  managing 
these  investments,  but  it  has  considerable  work  ahead  to  fully  implement 
mature  and  effective  processes.  We  are  making  recommendations  to 
strengthen  INS'  investment  management  capabihties. 

We  are  sending  copies  of  this  report  to  Senator  Judd  Gregg,  Chairman,  and 
Senator  Ernest  R  Hollings,  Ranking  Minority  Member,  Senate 
Appropriations  Subcommittee  on  Commerce,  Justice,  State,  and  Judiciary; 
Senator  Spencer  Abraham,  Chairman,  and  Senator  Edward  M.  Kennedy, 
Ranking  Minority  Member,  Senate  Judiciary  Subcommittee  on  Immigration; 
Representative  Harold  Rogers,  Chairman,  and  Representative  Jose  E. 
Serrano,  Ranking  Minority  Member,  House  Appropriations  Subcommittee 
on  Commerce,  Justice,  State,  and  the  Judiciary;  Representative  Lamar 
Smith,  Chairman,  and  Representative  Sheila  Jackson  Lee,  Ranking  Minority 
Member,  House  Judiciary  Subcommittee  on  Immigration  and  Claims;  the 
Honorable  Jacob  J.  Lew,  Director,  Office  of  Management  and  Budget;  and 
Mary  Ann  Wyrsch,  Acting  Commissioner  of  the  Immigration  and 
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Naturalization  Service.  Copies  will  also  be  made  available  to  others  upon 
request. 


Randolph  C.  Hite 
Director,  IT  Systems  Issues 


David  L.  McClure 

Director,  IT  Management  Issues 
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Executive  Summary 

Purpose 

The  Immigration  and  Naturalization  Service  (INS),  an  agency  of  the 
Department  of  Justice,  invests  hundreds  of  millions  of  dollars  each  year  in 
information  technology  (IT)  to  carry  out  its  core  missions  of  (1)  preventing 
aliens  from  entering  the  United  States  illegally  and  removing  aliens  ■who 
succeed  in  doing  so  and  (2)  providing  services  or  benefits  to  facilitate  entry, 
residence,  employment,  and  naturalization  of  legal  immigrants. 

The  Clinger-Cohen  Act  requires  agency  heads  to  implement  a  process  for 
maximizing  the  value  and  assessing  and  managing  the  risks  of  its  IT 
investments.  ^  Our  research  of  leading  private  and  public  sector 
organizations'  IT  management  practices  indicates  that  effective  investment 
management  requires  the  use  of  defined  and  disciplined  investment 
management  processes.^  Such  structured  processes  provide  a  systematic 
method  for  agencies  to  nunimize  risks  while  maximizing  the  return  on 
investments.  Given  the  importance  of  IT  investment  management  to  INS, 
GAO  determined  whether  (1)  INS  is  effectively  managing  its  IT  investments 
and  (2)  the  Department  of  Justice  is  effectively  promoting,  guiding,  and 
overseeing  INS'  investment  management  activities. 

Background 

Each  year  INS  invests  hundreds  of  millions  of  dollars  on  IT  systems  and 
activities.  According  to  INS,  in  fiscal  year  2000,  it  obligated  about 
$327  million  on  IT  activities,  including  about  $94  million  for  development 
and  deployment  and  the  remaining  amount  for  operations  and 
maintenance,  including  m^or  enhancements  to  existing  systems.  For  fiscal 
year  2001,  INS  plans  to  spend  about  $226  million  on  IT  for  operations  and 
maintenance  activities.® 

Recent  studies  have  identified  significant  weaknesses  in  INS'  management 
of  its  IT  resources.  In  August  1998,  the  Logistics  Management  Institute 
(LMI)  reported  that  INS  did  not  track  and  manage  projects  to  a  set  of  cost, 

^The  fiscal  year  1997  Omnibus  Consolidated  Appropriations  Act,  P.  L.  104-208,  renamed  both 
Division  D  (the  Federal  Acquisition  Reform  Act)  and  E  (the  Information  Technology 
Management  Reform  Act)  of  the  1996  DOD  Authorization  Act,  P.  L.  104-106,  as  the  Clinger- 
Cohen  Act  of  1996. 

"^Executive  Guide:  Improving  Mission  Performance  Through  Strategic  Information 
Management  and  Technology  (GAO/AIMD-94-115,  May  1994). 

WS  has  not  yet  decided  how  much  it  will  spend  in  fiscal  year  2001  on  IT  for  development 
and  deployment  activities. 
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schedule,  technical,  and  benefit  baselines.^  LMI  noted  that  while  INS  had 
defined  good  procedures  for  the  development  of  IT  projects,  it  did  not 
consistently  follow  them.  Similarly,  in  July  1999,  the  Justice  Inspector 
General  (IG)  reported  that  INS  was  not  adequately  managing  its 
information  sj^tems.®  In  particular,  the  IG  reported  that  (1)  estimated 
completion  dates  for  some  IT  projects  had  been  delayed  without 
explanation  for  the  delays,  (2)  project  costs  continued  to  spiral  upward 
with  no  justification  for  how  funds  are  spent,  and  (3)  projects  were  nearing 
completion  with  no  assurance  that  they  will  meet  performance  and 
functional  requirements.  More  recently,  in  August  2000,  GAO  reported  that 
INS  did  not  have  an  enterprise  architecture  (or  agency  wide  blueprint)  to 
guide  the  development  of  its  new  and  the  evolution  of  its  existing 
information  systems,  and  it  had  not  yet  established  the  management 
structure  and  controls  to  develop  one.®  An  enterprise  architecture  is  a 
Clinger-Cohen  Act  requirement  and  a  practice  of  successful  public  and 
private  sector  organizations.  Until  INS  has  such  an  architecture,  it  will  be 
unable  to  fully  ensure  that  the  hundreds  of  millions  of  dollars  it  spends 
each  year  on  new  and  existing  information  systems  will  optimally  support 
mission  needs.  As  a  result,  GAO  recommended  that  INS  develop  a  complete 
enterprise  architecture,  including  both  a  current  and  target  architecture 
and  a  plan  for  moving  between  the  two,  and  that  it  manage  the 
development  of  the  architecture  as  an  agency  wide  priority. 

The  Clinger-Cohen  Act  of  1996  was  enacted  to  address  longstanding 
problems  related  to  federal  IT  management.  Among  other  things,  it  requires 
agency  heads  to  implement  a  process  for  maximizing  the  value  and 
assessing  and  managing  the  risks  of  its  acquisitions.  A  key  goal  of  the 
Clinger-Cohen  Act  is  that  agencies  have  processes  and  information  in  place 
to  help  ensure  that  IT  projects  are  being  implemented  at  acceptable  costs, 
within  reasonable  and  expected  time  frames,  and  are  contributing  to 
tangible,  observable,  improvements  in  mission  performance. 


^Reengineering  Information  Technology  Management  at  the  Immigration  and 
Naturalization  Service,  Logistics  Management  Institute,  August  1998.  LMI  is  a  private, 
nonprofit  corporation  that  provides  management  consulting,  research,  and  analysis  to 
governments  and  other  nonprofit  organizations. 

^FoUow-up  Review:  Immigration  and  Naturalization  Service  Management  of  Automation 
Programs,  Office  of  the  Inspector  General,  Audit  Division,  U.S.  Department  of  Justice,  July 
1999. 

^Information  Technology:  INS  Need  to  Better  Manage  the  Development  of  Its  Enterprise 
Architecture  (GAO/AIMD-00-212,  August  1, 2000). 
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In  May  2000,  GAO  issued  an  Information  Technology  Investment 
Management  (TTIM)  maturity  framework,  which  identifies  critical 
processes  for  successful  IT  investment  and  organizes  these  processes  into 
a  framework  of  increasingly  mature  stages.  ITIM  supports  the 
fundamental  requirements  of  the  Clinger-Cohen  Act,  which  calls  for  IT 
investment  and  capital  planning  processes  and  IT  performance 
measurement.  ITIM  is  intended  to  provide  a  tool  for  implementing  these 
processes  incrementally  and  effectively.  ITIM  has  been  favorably  reviewed 
by  federal  Chief  Information  Officers  (CIOs)  and  members  of  GAO's 
advisory  council  on  IT  management. 

ITIM  is  a  hierarchical  model  comprising  five  different  maturity  stages.  Each 
stage  builds  upon  the  lower  stages  and  represents  a  step  toward  achieving 
both  stable  and  effective  IT  investment  management  processes.  With  the 
exception  of  the  first  stage — which  reflects  a  general  absence  of 
investment  management  processes — each  maturity  stage  is  composed  of 
critical  processes  that  must  be  implemented  and  institutionalized  for  the 
organization  to  satisfy  the  requirements  of  that  stage  and  be  able  to 
advance  to  the  next  stage.  These  critical  processes  are  further  broken 
down  into  key  practices.  Key  practices  are  the  specific  tasks  and 
conditions  that  must  be  in  place  for  an  organization  to  effectively 
implement  the  necessary  critical  processes.  Using  ITIM,  GAO  evaluated 
relevant  processes  in  maturity  stages  two  and  three.®  GAO  did  not  assess 
stages  four  and  five  because  INS  acknowledged  that  it  did  not  have  any 
stage  four  and  five  capabilities.  Figure  1  shows  the  five  ITIM  stages  and  a 
brief  description  of  each  stage. 


^Information  Technology  Investment  Management:  A  Framework  for  Assessing  and 
Improving  Process  Maturity  (Exposure  Draft)  (GAO/AIMD-10.1.23,  May  2000). 

'Stage  two  critical  processes  are  IT  investment  board  operations,  IT  project  oversight,  IT 
asset  tracking,  business  needs  identification  for  IT  projects,  and  proposal  selection.  Stage 
three  critical  processes  that  GAO  reviewed  are  portfolio  selection  criteria  definition, 
investment  analysis,  portfolio  development,  and  portfolio  performance  oversight. 
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Figure  1 :  The  Five  Stages  of  Maturity  Within  ITIM 
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;  Description _ 

Investment  benchmarking  and  IT-enabled 
change  management  techniques  are  deployed 
to  strategically  shape  business  outcomes. 


Process  evaluation  techniques  focus  on 
Improving  the  performance  and  management 
of  the  organization's  IT  Investment  portfolio. 


Comprehensive  IT  investment  portfolio  selection 
and  control  techniques  are  in  place  that 
incorporate  benefit  and  risk  criteria  linked  to 
mission  goals  and  strategies. 


Repeatable  investment  control  techniques  are  in 
place,  and  the  key  foundation  capabilities  have 
been  implemented  focusing  on  cost  and  schedule 
activities. 


There  is  little  awareness  of  investment 
management  techniques.  IT  management 
processes  are  ad  hoc,  project-centric,  and 
have  widely  variable  outcomes. 


Results  in  Brief  INS  has  limited  capability  to  effectively  manage  its  planned  and  ongoing  IT 

investments.  To  its  credit,  INS  has  some  important  IT  investment 
management  capabilities  in  place  to  build  upon  and  establish  effective 
investment  management  processes.  However,  it  has  considerable  work 
ahead  to  fully  implement  mature  and  effective  processes.  Until  INS  fully 
implements  such  processes,  it  will  not  know  whether  it  is  making  the  best 
investment  decisions  to  optimize  mission  performance,  whether  its 
selected  mix  of  investments  best  meets  its  overall  mission  and  business 
priorities,  or  whether  it  is  adequately  managing  the  risks  associated  with 
these  investments. 

The  first  mqjor  step  to  building  a  soimd  IT  investment  management  process 
is  to  be  able  to  measure  the  progress  of  existing  IT  projects  to  identify 
variances  in  cost,  schedule,  and  performance  expectations,  and  take 
corrective  action,  if  appropriate,  and  to  establish  basic  capabilities  for 
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selecting  new  IT  proposals.  INS  has  made  some  progress  in  establishing 
basic  selection  capabilities.  For  example,  INS  has  an  investment  review 
board  (IRB),  which  is  comprised  of  both  IT  and  business  senior  executives 
and  functions  as  INS'  central  decision-making  body  for  IT  projects. 
However,  INS  has  not  yet  implemented  investment  control  processes  and 
thus  does  not  know  if  its  IT  projects  are  meeting  cost,  schedule,  and 
performance  expectations.  For  example,  INS  executives  do  not  regularly 
track  and  monitor  the  progress  of  INS'  projects  toward  achieving  stated 
commitments  by  comparing  up-to-date  progress  data  with  expectations. 
Without  this  information,  INS  executives  do  not  have  adequate  assurance 
that  IT  projects  are  being  developed  on  schedule  and  within  budget,  and 
whether  INS'  investments  will  deliver  promised  capabilities  and  benefits. 

The  second  major  step  toward  effective  IT  investment  management 
requires  that  an  organization  continually  assess  proposed  and  ongoing 
projects  as  an  integrated  and  competing  set  of  investment  options.  This 
enables  the  organization  to  consider  the  relative  costs,  benefits,  and  risks 
of  new  proposals  along  with  previously  funded  investments  and  identify 
the  appropriate  mix  of  IT  investments  that  best  meets  its  mission, 
strategies,  and  goals.  However,  INS  has  not  yet  implemented  a  process  to 
compare  both  proposed  and  ongoing  IT  investments  to  determine  priorities 
and  to  make  decisions  about  what  projects  to  fund  based  on  their  relative 
costs,  benefits,  schedule,  and  risks.  As  a  result,  INS  executives  are  imable 
to  assess  and  make  trade-offs  about  the  relative  merits  of  spending  funds  to 
develop  new  systems,  enhance  current  systems,  or  continue  operating  and 
maintaining  existing  systems. 

Further,  the  Department  of  Justice  has  a  vital  leadership  role  to  play  in 
ensuring  that  its  component  agencies,  like  INS,  have  effective  IT 
investment  management  capabilities.  However,  Justice  has  not  issued  any 
directive  to  its  bureaus,  including  INS,  on  the  need  to  institutionalize 
effective  IT  investment  management  capabilities,  nor  has  it  issued 
gmdance  on  how  to  accomplish  this,  and  it  has  not  provided  oversight  on 
its  bureaus'  investment  management  development  efforts.  During  the 
course  of  our  work,  however.  Justice  began  drafting  IT  investment 
management  policy  and  guidance.  Justice  officials  stated  that  they  plan  to 
issue  the  final  policy  by  the  end  of  December  2000  and  the  guidance  by 
March  2001. 
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Principal  Findings 


INS  Lacks  Foundation 
Capabilities  Upon  Which  to 
Build  IT  Investment 
Management  Maturity 


To  develop  overall  sound  IT  investment  management  capabilities,  an 
organization  must  first  be  able  to  control  its  investments  so  that  they  finish 
predictably  within  established  schedule  and  budget  e3q)ectations  and  to 
establish  basic  capabilities  for  selecting  new  IT  proposals.  To  INS'  credit,  it 
has  made  some  progress  in  establishing  basic  selection  capabilities.  For 
example,  INS  has  established  an  IRB,  which  comprises  both  IT  and 
business  senior  executives.  The  IRB  functions  as  INS'  central  decision¬ 
making  body  for  IT  projects  and  has  broad  support  across  the  agency  for  its 
investment  decisions.  In  addition,  the  IRB  has  followed  a  structured 
process  for  developing  and  selecting  new  IT  proposals. 


INS  has  not  yet  implemented  investment  control  processes  needed  to 
adequately  ensure  that  its  IT  projects  meet  established  cost  and  schedule 
expectations.  In  particular,  INS  has  not  (1)  consistently  developed  and 
maintained  project  management  plans  that  include  cost  and  schedule 
controls,  (2)  regularly  tracked  and  monitored  its  IT  projects'  performance 
to  determine  whether  they  are  meeting  their  cost  and  schedule 
e3q)ectations,  and  (3)  acted  to  address  identified  cost  and  schedule 
performance  problems.  In  addition,  INS  has  not  clearly  identified  business 
needs  for  each  of  its  projects  or  trained  IT  project  staff  in  business  needs 
identification. 


According  to  INS,  this  lack  of  effective  investment  control  capabilities 
exists  because  it  has  not  viewed  the  need  for  them  as  an  institutional 
priority.  This  is  evidenced  by  the  fact  that  INS  is  still  e3q)eriencing  some  of 
the  same  weaknesses  that  were  identified  in  earlier  reviews.  For  example, 
INS  is  still  not  consistently  developing  and  maintaining  project 
management  plans  and  regularly  tracking  its  IT  projects'  performance  to 
determine  whether  they  are  meeting  their  cost  and  schedule  ejqjectations. 
As  a  result,  INS'  limited  investment  control  capabilities  significantly 
increase  the  chances  that  its  IT  projects  wOI  be  late,  cost  more  than 
expected,  not  perform  as  intended,  and  not  deliver  promised  business 
value. 
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INS  Is  Not  Managing  IT 
Investments  as  a  Complete 
Portfolio 


Justice  Is  Not  Guiding  and 
Overseeing  INS'  Investment 
Management  Approach 


Once  new  proposals  can  be  selected  and  developed  on  schedule  and  on 
budget,  organizations  need  to  continually  assess  and  manage  all  of  their  IT 
projects  (i.e.,  projects  that  are  proposed,  imder  development,  and  in 
operation)  based  on  expected  cost,  benefits,  schedule,  and  risk  to  create  a 
complete  strategic  investment  portfolio.  Taking  such  a  portfolio 
perspective  enables  the  organization  to  consider  its  investments 
comprehensively,  considering  new  proposals  along  with  previously  funded 
investments,  and  identifying  the  mix  of  IT  investments  that  best  meet  its 
mission  needs. 

However,  INS  is  not  effectively  managing  its  IT  investments  as  a  complete 
portfolio.  While  INS  has  defined  portfolio  categories  and  assigned  each 
investment  (including  new  and  ongoing  investments)  to  one  of  these 
categories,  it  has  not  defined  the  cost,  benefits,  schedule,  and  risk  criteria 
to  best  support  its  mission  and  business  priorities,  and  it  does  not  use  these 
criteria  to  select  IT  projects  for  funding.  Without  the  use  of  such  criteria, 
INS  lacks  critical  information  to  examine  the  mix  of  new  proposals  and 
ongoing  investments  within  and  across  its  investment  portfolios  in  order  to 
select  those  investments  that  best  align  with  mission  needs  and  priorities. 
Further,  INS  executives  have  not  monitored  the  performance  of  each  of 
INS'  IT  investments  in  its  portfolio  by  comparing  actual  cost,  benefits, 
schedule,  and  risk  data  against  expectations.  According  to  INS,  it  has  not 
established  these  investment  management  capabilities  because  IT 
investment  management  has  not  been  an  institutional  priority.  In  the 
absence  of  such  investment  management  capabilities,  INS  is  unable  to 
consider  the  relative  merits  of  all  investments,  including  both  new  and 
ongoing,  to  select  those  investments  that  best  meet  its  mission  needs  and 
priorities. 


The  Clinger-Cohen  Act  requires  that,  among  other  things,  the  head  of  each 
agency  implement  a  process  for  maximizing  the  value  of  the  agency's  IT 
investments  and  assessing  and  managing  the  risks  of  its  IT  investments, 
and  that  the  agency  CIO  work  with  the  agency  head  in  implementing  such  a 
process.  However,  Justice  has  not  provided  INS,  or  any  other  Justice 
component,  direction,  guidance,  and  oversight  on  IT  investment 
management  activities.  According  to  Justice  officials.  Justice  had  not  done 
so  because  of  other  competing  department  priorities,  even  though  the 
department  and  its  components  spent  about  $3  billion  on  IT  in  fiscal  years 
1999  and  2000. 
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During  the  course  of  our  work,  Justice  began  drafting  IT  investment 
management  policy  and  guidance  documents  in  collaboration  with  an 
intercomponent  working  group.  The  draft  policy  directs  Justice 
components  to  establish  and  use  an  IT  investment  man^ement  process 
and  directs  the  Justice  CIO  to  monitor  the  components'  investment 
management  processes  through  periodic  briefings.  A  supplemental 
guidance  dociunent  provides  procedures  for  developing  an  investment 
management  process.  Justice  officials  stated  that  they  plan  to  issue  the 
final  policy  by  the  end  of  December  2000  and  the  guidance  by  March  2001. 
lA^thout  effective  guidance  and  oversight,  Justice  does  not  have  adequate 
assurance  that  INS,  as  well  as  its  other  components,  have  the  necessary 
investment  management  processes  in  place  to  maximize  the  value  of  their 
rr  investments  and  manage  the  risks  associated  with  them. 


Recommendations 
Executive  Action 


To  strengthen  INS'  investment  management  capability  and  address  the 
weaknesses  discussed  in  this  report,  GAO  recommends  that  you  direct  the 
Commissioner  of  INS  to  designate  development  and  implementation  of 
effective  IT  investment  management  processes  as  an  agencywide  priority 
and  manage  it  as  such.  Specifically,  you  should  direct  the  Commissioner  to 
do  the  following: 


•  Develop  a  plan,  within  9  months,  for  implementing  IT  investment 
management  process  improvements  that  is  based  on  stages  two  and 
three  critical  processes  and  specifies  measurable  goals  and  time  frames, 
ranks  initiatives,  defines  a  management  structure  for  directing  and 
controlling  the  improvements,  establishes  review  milestones,  and 
recognizes  any  direction  and  guidance  that  Justice  issues.  This  plan 
should  first  focus  on  those  critical  processes  in  stage  two  of  ITIM 
because,  collectively,  they  provide  the  foundation  for  building  a  mature 
IT  investment  management  process. 

•  Submit  the  plan  to  the  Justice  CIO  for  review  and  approval. 

•  Implement  the  approved  plan  and  report  to  the  Justice  CIO,  according  to 
established  review  milestones,  on  progress  made  against  the  plan's 
goals  and  time  frames. 


Further,  because  the  absence  of  effective  investment  management 
processes  and  an  enterprise  architecture®  severely  limits  INS'  ability  to 


^Information  Technology:  INS  Needs  to  Better  Manage  the  Development  of  Its  Enterprise 
Architecture  (GAO/AIMD-00-212,  August  1, 2000). 
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effectively  manage  its  IT  investments,  GAO  recommends  that  imtil  INS 
develops  a  complete  enterprise  architecture  and  implements  the  key 
practices  associated  with  stages  two  and  three  critical  processes,  as 
described  in  this  report,  you  direct  the  Commissioner  to  limit  requests  for 
future  appropriations  for  IT  only  to  efforts  that 

•  support  ongoing  operations  and  maintenance,  but  not  mqjor 
enhancements,  of  existing  systems; 

•  support  INS  efforts  to  develop  and  implement  IT  investment 
management  processes  and  an  enterprise  architecture; 

•  are  small,  represent  low  technical  risk,  and  can  be  delivered  in  a 
relatively  short  period  of  time;  or 

•  are  congressionally  mandated. 

Further,  to  improve  Justice's  guidance  and  oversight  of  components'  IT 
investment  management  process  activities,  GAO  also  recommends  that  you 
direct  the  Justice  CIO  to  follow  through  on  the  department's  plans  to  issue 
an  IT  investment  management  policy  and  guidance  to  the  components  and 
to  ensure  that  the  policy  and  guidance 

•  directs  Justice  components  and  bureaus,  including  INS,  to  develop  and 
implement  IT  investment  management  processes. 

•  instructs  Justice  components  and  bureaus  on  how  to  develop  an 
investment  management  process.  This  guidance  should  be  based  on  the 
investment  management  guidance  contained  in  this  report  and,  at  a 
minimum,  should  include  component  roles,  responsibilities,  authorities, 
and  policies  and  procedures  for  developing  an  IT  investment 
management  process. 

•  directs  the  Justice  CIO  to  monitor  the  components'  progress  in 
developing  and  establishing  an  IT  investment  management  process  and 
take  appropriate  action  if  they  are  not  progressing  sufficiently. 


In  written  comments  on  a  draft  of  this  report.  Justice  generally  agreed  with 
our  recommendations.  However,  it  offered  minor  wording  modifications  on 
two  recommendations  that  it  said  would  increase  its  ability  to  fully 
implement  them.  Justice  also  disagreed  with  our  finding  that  Justice  is  not 
guiding  and  directing  INS'  investment  management  approach. 

Justice  generally  agreed  with  our  recommendation  that  INS  develop  and 
submit  to  Justice  a  plan  for  implementing  investment  management  process 
improvements.  However,  Justice  suggested  that  the  time  frame  for 
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developing  the  plan  be  clarified  such  that  INS  has  6  months  to  develop  and 
submit  its  plan  to  Justice  once  Justice  issues  its  new  IT  investment 
management  guidance.  Because  GAO's  recommendation  directed  INS  to 
consider  any  Justice  guidance  and  direction  in  developing  its  investment 
management  process  improvement  plan,  GAO  has  modified  the 
recommendation  to  include  an  additional  3  months  to  allow  time  for 
Justice  to  issue  its  guidance,  which  it  plans  to  do  m  March  2001. 

Justice  also  concurred  with  GAO's  recommendation  for  INS  to  limit  future 
appropriation  requests  for  FT  to  certain  investment  categories  because  it 
lacks  an  enterprise  architecture  and  effective  investment  management 
processes,  but  it  suggested  that  GAO  specify  that  this  recommendation  is  in 
effect  imtil  INS  completes  its  architecture  and  implements  investment 
management  processes.  Because  this  is  the  intent  of  GAO's 
recommendation,  GAO  has  clarified  the  recommendation  to  make  this 
explicit. 

Further,  while  INS  agreed  with  GAO's  recommendation  for  Justice  to  issue 
an  investment  management  policy  and  guidance  to  its  components, 
including  INS,  it  disagreed  with  GAO's  finding  that  Justice  is  not  guiding 
and  directing  INS'  investment  management  approach.  According  to  Justice, 
it  has  established  guidance  for  all  aspects  of  IT  management  that  its 
components  are  expected  to  follow  and  has  a  process  for  overseeing 
components'  management  of  their  investments.  To  support  its  position. 
Justice  cited  several  examples,  such  as  Justice  approval  authority  of  all 
component  FT  investments  with  life-cycle  cost  over  $1  million.  Justice 
establishment  of  an  IT  Investment  Board,  and  Justice  meetings  with 
components. 

GAO  does  not  agree  with  Justice's  position.  While  GAO  concurs  that  the 
examples  cited  by  Justice  represent  important  IT  management  functions  to 
be  performed  in  providing  management  oversight  of  individual  IT 
investments,  such  management  oversight  is  not  the  focus  of  GAO's 
findings,  conclusions,  and  recommendations.  Rather,  GAO's  report 
addresses  Justice's  efforts  to  ensure  that  its  components,  including  INS, 
have  each  defined  and  implemented  effective  IT  investment  management 
processes.  As  such,  GAO  sought  evidence  from  Justice  demonstrating  that 
it  has  directed  its  components  to  establish  such  processes,  provided 
guidance  to  its  components  on  how  to  develop  and  implement  these 
processes,  and  monitored  its  components'  progress  to  determine  whether 
they  are  implementing  such  processes.  However,  besides  the  steps  that 
Justice  initiated  during  the  course  of  GAO  inquiries  and  plans  to  take. 
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which  GAO  has  described  in  this  report,  GAO  found  no  such  evidence. 
Moreover,  Justice  stated  in  its  vmtten  comments  that  it  agreed  with  GAO's 
recommendation  for  it  to  provide  investment  management  process 
direction,  guidance,  and  oversight  to  its  components. 

Justice's  written  comments  are  discussed  in  further  detail  in  chapter  5,  and 
the  full  text  of  its  comments  is  reproduced  in  appendix  I. 
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The  mission  of  INS,  an  agency  of  the  Department  of  Justice,  is  to 
administer  and  enforce  the  immigration  laws  of  the  United  States.  To 
accomplish  this,  INS  is  organized  into  three  core  business  areas — 
enforcement,  immigration  services,  and  corporate  services.  Enforcement 
includes,  among  other  things,  conducting  inspections  of  travelers  entering 
the  United  States  as  they  arrive  at  more  than  300  land,  sea,  and  air  ports  of 
entry;  detecting  and  preventing  the  smuggling  and  illegal  entry  of  aliens; 
and  identifying  and  removing  persons  who  have  no  lawful  immigration 
status  in  the  United  States.  Immigration  services,  which  involve  regulating 
permanent  and  temporary  iimnigration  to  the  United  States,  include 
granting  legal  permanent  residence  status,  nonimmigrant  status  (e.g., 
tourists  and  students),  and  naturalization.  Corporate  services  include 
records  management,  financial  management,  personnel  management,  and 
inventory  management  support  for  INS  activities. 

INS'  IT  assets  play  a  significant  role  in  (1)  receiving  and  processing 
naturalization  and  other  benefit  applications,  (2)  processing  immigrants 
and  nonimmigrants  entering  and  leaving  the  United  States,  and 
(3)  identifying  and  removing  people  who  have  no  lawful  immigration  status 
in  the  United  States.  For  example,  the  Computer-Linked  Application 
Information  Management  System  (CLAIMS  4)  is  a  centralized  case 
management  tracking  system,  that  offers  support  for  a  variety  of  tasks 
associated  with  processing  and  ac^udicating  naturalization  benefits.  In 
addition,  the  Deportable  Alien  Control  System  (DACS)  automates  many  of 
the  functions  associated  with  tracking  the  location  and  status  of  illegal 
aliens  in  removal  proceedings,  including  detention  status. 


INS'  Current  IT 
Investment  Efforts 


INS  has  multiple  efforts  underway  to  develop  and  acquire  new  information 
systems  and  to  maintain  existing  ones.  According  to  INS,  in  fiscal  year 
2000,  it  obligated  about  $327  million  on  IT  activities,  including  about 
$94  million  for  new  development  and  the  remaining  amount,  which 
includes  enhancing  existing  systems,  for  operations  and  maintenance.  For 
example,  INS  obligated  $14.5  million  in  fiscal  year  2000  to  continue 
development  of  CLAIMS  4,  which  supports  the  processing  of  applications 
and  petitions  for  immigrant  benefits  and  is  intended  to  fully  replace 
CLAIMS  3.  In  addition,  INS  obligated  about  $18  million  in  fiscal  year  2000  to 
further  deploy  its  Integrated  Surveillance  Intelligence  System  (ISIS),  which 
includes  the  deployment  of  intelligent  computer  aided  detection  systems, 
imattended  ground  sensors,  and  fixed  cameras  along  the  northern  and 
southern  borders  to  provide  aroimd-the-clock  visual  coverage  of  the 
border.  For  fiscal  year  2001,  INS  plans  to  spend  about  $226  million  on  IT  for 


Page  20 


GAO-01-146  INS’  IT  Investments 


Chapter  1 
Introduction 


operations  and  maintenance  activities.^  INS  funds  most  of  its  IT  efforts 
with  operation  and  maintenance  funds  and  currently  is  developing  or 
maintaining  74  information  systems. 


Recent  Reviews  Have 
Identified  IT  Project 
Management 
Weaknesses 


Recent  reviews  have  identified  several  weaknesses  in  INS'  management  of 
its  IT  projects.  For  example,  in  August  1998,  the  Logistics  Management 
Institute  (LMI)^  reported  that  INS'  Office  of  Information  Resources 
Management  (OIRM)  (1)  did  not  maintain  accmate  cost  estimates  for  the 
complete  life  cycle  of  projects  and  (2)  did  not  track  and  manage  projects  to 
a  set  of  cost,  schedule,  technical,  and  benefit  baselines.^  Further,  LMI  noted 
that  while  INS'  System  Development  Life  Cycle  (SDLC)  manual  provides  a 
good  model  for  systems  development  projects,  OIRM  did  not  consistently 
follow  it,  often  bypassing  key  SDLC  phases.  ^ 


Similarly,  in  July  1999,  the  Justice  Inspector  General  (IG)  reported  that 
(1)  estimated  completion  dates  for  some  INS  IT  projects  had  been  delayed 
without  explanation  for  the  delays,  (2)  project  costs  continued  to  spiral 
upward  with  no  justification  for  how  funds  are  spent,  and  (3)  projects  were 
nearing  completion  with  no  assurance  that  they  would  meet  performance 
and  functional  requirements.  ® 


Recognizing  the  need  to  address  these  weaknesses,  INS  established  an 
Operational  Assessment  Team  to  analyze  reported  weaknesses  and 
recommend  specific  actions  to  address  them.  The  Operational  Assessment 
Team  validated  the  deficiencies  identified  in  the  LMI  and  Justice  IG  reports 
and  identified  additional  ones.  For  example,  the  team  found  that  system 


has  not  yet  decided  how  much  it  will  spend  in  fiscal  year  2001  on  IT  for  development 
and  deployment  activities. 

^LMI  is  a  private,  nonprofit  corporation  that  provides  management  consulting,  research,  and 
analysis  to  governments  and  nonprofit  organizations. 

^Reenffineering  Information  Technology  Management  at  the  Immigration  and 
Naturalization  Service,  Logistics  Management  Institute,  August  1998. 

^System  development  life  cycle”  is  a  term  used  to  refer  to  the  phases  of  a  system's 
development  firom  beginnmg  to  end  (i.e.,  firom  perceived  need  for  a  system  extending 
through  systems  design,  development,  implementation,  operations,  and  maintenance). 

^Follow-up  Review:  Immigration  and  Naturalization  Service  Management  of  Automation 
Programs,  Office  of  the  Inspector  General,  Audit  Division,  U.S.  Department  of  Justice,  July 
1999. 
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requirements  were  not  consistently  collected,  recorded,  documented, 
tracked,  and  controlled.  To  illustrate,  of  106  projects  reviewed  by  the  team, 
fewer  than  60  percent  had  docmnented  requirements  and  most  of  the 
requirements  that  had  been  documented  were  not  current. 

Further,  in  August  2000,  we  reported  that  INS  did  not  have  an  enterprise 
architecture  to  guide  the  development  and  evolution  of  its  information 
systems.®  An  enterprise  architecture  is  an  institutional  systems  blueprint 
that  defines  in  both  business  and  technological  terms  the  organization's 
current  and  target  operating  environments  and  provides  a  road  map  for 
moving  from  one  to  the  other.  It  is  required  by  the  Clinger-Cohen  Act  and  is 
a  recognized  practice  of  successful  public  and  private  sector  organizations. 
INS  had  initiated  some  limited  efforts  to  document  its  current  architecture, 
but  it  had  not  yet  begun  developing  a  target  architecture  or  a  plan  to  move 
from  the  current  to  the  target  envirorunent.  Moreover,  INS  had  not  yet 
established  the  management  structure  and  controls  to  develop  the 
architecture.  The  absence  of  such  an  enterprise  architecture  increases  the 
risk  that  the  hundreds  of  millions  of  dollars  INS  spends  each  year  on 
information  systems  will  not  be  well  integrated  or  compatible  and  will  not 
effectively  support  mission  needs  and  priorities. 


Overview  of  INS' 
Current  Approach  to  IT 
Investment 
Management 


In  1997,  INS  established  an  investment  review  board  (IRB).  The  IRB 
consists  of  four  voting  members — ^the  Deputy  Commissioner  (Chair)  and 
INS'  three  Executive  Associate  Commissioners — and  advisory  or 
supporting  members,  including  the  Director  of  the  Budget  Office  and  the 
Acting  Associate  Commissioner  of  the  Office  of  Information  Resources 
Management.  In  November  1998,  INS  also  established  the  Executive 
Steering  Committee  (ESC)  to  support  the  IRB.  The  ESC  comprises 
portfolio  managers  and  advisory  members,  which  analyze  investment 
proposals  and  make  recommendations  on  these  proposals  to  the  IRB.  ^ 


The  IRB  has  established  a  process  for  selecting  new  IT  proposals. 
According  to  INS  officials,  new  proposals  are  developed  throughout  the 


^Information  Technology:  INS  Needs  to  Better  Manage  the  Development  of  Its  Enterprise 
Architecture  (GAO/AIMD-00-212,  August  1,  2000). 

^Portfolio  managers  are  individuals  who  are  responsible  for  managing  a  group  of  systems 
within  a  particular  business  area  or  portfolio.  INS  has  defined  eight  portfolio  categories: 
Biometrics,  Corporate,  Enforcement,  Examination,  Infrastructure,  Inspections,  IRM 
Operations,  and  Management. 
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year  as  business  needs  are  identified  and  are  forwarded  to  the  appropriate 
portfolio  manager  for  review.  After  reviewing  the  proposal,  the  portfolio 
manager  forwards  it  to  the  ESC  for  consideration  for  funding.  The  ESC 
examines  the  proposals  submitted  and  determines  the  appropriate  funding 
for  each  project.  Once  funding  is  determined,  the  ESC  forwards  the 
proposed  funding  levels  to  the  IRB,  which  makes  the  final  investment 
selections  and  budget  formulation  decisions.  See  figure  2  for  INS'  new 
proposal  selection  process. 


Figure  2:  Current  INS  New  IT  Proposal  Selection  Process 


Source:  INS. 

As  part  of  INS'  annual  budget  execution  process,  the  IRB  considers  the 
funding  requests  of  ongoing  and  new  projects.  Project  managers  define 
requirements  for  their  ongoing  projects,  which  they  submit  to  the 
responsible  portfolio  managers  for  review.  After  reviewing  the 
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requirements  and  funding  requests,  each  portfolio  manager  submits  them 
to  the  ESC  for  review  and  to  the  IRB  for  approval.  The  approved  funding  is 
submitted  to  the  Budget  Office  for  inclusion  into  its  budget  execution 
process.  According  to  INS  officials,  new  proposals  are  considered  for 
funding  only  after  ongoing  projects  have  been  funded. 


Framework  for 
Assessing  Agencies'  IT 
Investment 
Management 


Several  recent  management  reforms — including  the  revision  to  the 
Paperwork  Reduction  Act  and  the  passage  of  the  Clinger-Cohen  Act  of 
1996,  the  Government  Performance  and  Results  Act  of  1993,  and  the  Chief 
Financial  Officers  Act  of  1990 — have  introduced  requirements  emphasizing 
the  need  for  federal  agencies  to  improve  their  management  processes  for 
selecting  and  managing  IT  resources.  In  particular,  the  Clinger-Cohen  Act 
requires  that  the  head  of  each  agency  implement  a  process  for  maximizing 
the  value  of  the  agency’s  IT  investments  and  for  assessing  and  managing 
the  risks  of  its  acquisitions.  A  key  goal  of  the  Clinger-Cohen  Act  is  that 
agencies  have  processes  and  information  in  place  to  help  ensure  that 
projects  are  being  implemented  at  acceptable  costs  within  reasonable  and 
expected  time  frames  and  that  they  are  contributing  to  tangible,  observable 
improvements  in  mission  performance. 


We  and  the  Office  of  Management  and  Budget  (0MB)  have  developed 
guidance  to  assist  federal  agencies  in  managing  IT  investments.  One  such 
guide.  Assessing  Risks  and  Returns:  A  Guide  for  Evaluating  Federal 
Agencies'  IT  Investment  Decision-making,  incorporates  our  analysis  of 
the  management  practices  of  leading  private  and  public  sector 
organizations  as  well  as  the  provisions  of  mqjor  federal  legislation  (e.g., 
Clinger-Cohen  Act)  and  executive  branch  guidance  that  address  investment 
decision-making.®  The  guide  provides  a  method  for  determining  how  well  a 
federal  agency  is  selecting  and  managing  its  IT  resomces  and  identifies 
specific  areas  where  improvements  can  be  made. 

To  enhance  this  guidance,  we  issued  an  Information  Technology 
Investment  Management  (ITIM)  maturity  framework  in  May  2000.®  ITIM 
provides  a  common  framework  for  assessing  IT  capital  planning  and 
investment  management  practices  by  describing  the  organizational 


«GAO/AIMD-10.1.13,  February  1997. 

^Information  Technology  Investment  Management:  A  Framework  for  Assessing  and 
Improving  Process  Maturity  (Exposure  Draft)  (GAO/AIMD-10. 1.23,  May  2000). 
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processes,  and  their  interrelationships  that  are  the  tenets  of  good 
investment  management.  ITIM  is  based  on  the  best-practices  work  done  as 
part  of  our  ongoing  research  into  the  IT  management  practices  of  leading 
organizations. 

ITIM  is  a  hierarchical  model  comprising  five  maturity  stages.  These 
maturity  stages  represent  steps  toward  achieving  stable  and  mature 
investment  management  processes.  As  agencies  advance  through  the 
model's  stages,  their  capability  to  manage  IT  increases.  Each  stage  builds 
upon  the  lower  stages  and  enhances  the  organization's  ability  to  manage  its 
investments.  Mth  the  exception  of  the  first  stage,  each  maturity  stage  is 
composed  of  critical  processes  that  must  be  implemented  and 
institutionalized  for  the  organization  to  satisfy  the  requirements  of  that 
stage.  These  critical  processes  are  further  broken  down  into  key  practices 
that  describe  the  types  of  activities  that  an  agency  should  be  engaged  in  to 
successfully  implement  each  critical  process.  An  organization  that  has 
these  critical  processes  in  place  is  in  a  better  position  to  successfully  invest 
in  IT.  (See  figure  3  for  the  five  stages  and  associated  critical  processes). 
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Figure  3:  The  (TIM  Stages  of  Maturity  With  Critical  Processes 


Maturity  stages 


As  established  by  the  model,  each  critical  process  contains  five  core 
elements  that  indicate  whether  the  implementation  and  institutionalization 
of  a  process  can  be  effective  and  repeated.  The  five  core  elements  are: 

•  Purpose:  This  is  the  primary  reason  for  engaging  in  the  critical  process 
and  states  the  desired  outcome  for  the  critical  process. 

•  Organizational  commitment:  This  comprises  management  actions  that 
ensure  that  the  critical  process  is  established  and  will  endure.  Key 
practices  typically  involve  establishing  organizational  policies  and 
engaging  senior  management  sponsorship. 

•  Prerequisites:  These  are  the  conditions  that  must  exist  within  an 
organization  to  successfully  implement  a  critical  process.  This  typically 
involves  allocating  resources,  establishing  organizational  structures, 
and  providing  training. 

•  Activities:  These  are  the  key  practices  necessary  to  implement  a  critical 
process.  An  activity  occms  over  time  and  has  recognizable  results.  Key 
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practices  typically  involve  establishing  procedures,  performing  and 
tracking  the  work,  and  taking  corrective  actions  as  necessary. 

•  Evidence  of  performance:  This  comprises  artifacts,  documents,  or  other 
evidence  that  supports  a  contention  that  the  key  practices  within  a 
critical  process  have  or  are  being  implemented.  This  core  element 
typically  consists  of  the  collection  and  verification  of  physical, 
documentary,  or  testimonial  evidence  and  typically  involves  reviews  by 
objective  parties. 

With  the  exception  of  the  purpose  core  element,  each  of  the  other  core 
elements  contains  key  practices.  The  key  practices  are  the  attributes  and 
activities  that  contribute  most  to  the  effective  implementation  and 
institutionalization  of  a  critical  process.  (Figure  4  shows  the  relationship 
between  the  various  ITIM  components.) 


Figure  4:  ITIM  Component  Relationships 


Maturity 

stage 


Critical 

process 


organized  by 


Core 

element 


Key 

practice 


The  five  maturity  stages  represent  the  steps 
toward  achieving  a  maturp,  comprehehsive 
IT  Investment  management  process 


With  the  exception  of  Stage  1 ;  each  maturity 
stage  is  composed  of  crttical  processes^  which 
must  be  implemented  to  attain  that  stage. 


The  core  elements  provide  the  common  framework 
for  each  critical  process^  The  five  types  of  core 
elements  are  purpose,  drgani2ational  comhiitm^ 
jDrerequisites,  activities,  and  eviderice  of  performance^ 


Key  practices  are  the  tasks  within  a  core 
element  that  mu^  be  performed  by  an  " 
organization  in  order  to  effectively  implement 
and  institutionalize  a  criticai  process^ 
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Objectives,  Scope,  and 
Methodology 


Our  objectives  were  to  determine  whether  (1)  INS  is  effectively  managing 
its  IT  investments  and  (2)  the  Department  of  Justice  is  effectively 
promoting,  guiding,  and  overseeing  INS'  investment  management  activities. 


To  determine  whether  INS  is  effectively  managing  its  investments,  we 
applied  our  ITIM  framework  and  the  associated  assessment  method.  As 
part  of  the  ITIM  assessment  method,  INS  conducted  a  self-assessment  of  its 
IT  investment  management  activities  using  the  ITIM  framework.  In  its  self- 
assessment,  INS  indicated  whether  it  executed  each  of  the  key  practices  in 
stages  two  through  five.  INS  asserted  that  it  executed  many  of  the  key 
practices  within  stages  two  and  three  but  only  four  key  practices  in  all  of 
stages  four  and  five.  Accordingly,  we  did  not  include  ITIM  stages  four  and 
five  in  the  scope  of  our  review.  Also,  we  did  not  evaluate  the  key  practices 
within  stages  two  and  three  that  INS  stated  it  had  not  executed. 


We  evaluated  INS  against  9  of  the  10  critical  processes  in  stages  two  and 
three.  We  did  not  evaluate  INS  against  the  stage  three  critical  process 
Authority  Alignment  of  IT  Investment  Boards.  This  critical  process  is  only 
relevant  if  an  organization  has  more  than  one  IT  investment  board  and  INS 
has  only  one.  The  nine  critical  processes  we  examined  focus  primarily  on 
INS'  ability  to  effectively  select  and  control  its  IT  investments. 

To  determine  whether  INS  had  implemented  these  nine  critical  processes, 
we  evaluated  policies,  procedures,  and  guidance  related  to  INS'  IT 
investment  management  activities.  In  particular,  we  analyzed  the  following: 
organizational  charters,  INS'  System  Development  Life  Cycle  manual, 
requirements  management  process  guide,  and  administrative  manuals  (e.g.. 
Personal  Property  Handbook').  We  also  reviewed  documentation 
associated  with  specific  investment  management  activities,  such  as  IRB 
and  ESC  meeting  minutes,  project  management  plans,  system  deployment 
plans,  budget  formulation  and  execution  plans,  quarterly  reports  to  Justice, 
and  contractor  statements  of  work. 


In  addition,  we  reviewed  foirr  IT  projects  to  verify  the  execution  of  INS- 
defined  processes,  procedures,  and  practices.  The  four  projects  were 
selected  based  on  the  following  criteria:  (1)  the  projects  should  represent 
different  life  cycle  phases  (e.g.,  requirements  definition,  design,  operations 
and  maintenance),  (2)  the  projects  should  support  different  INS  business 
areas  (e.g..  Examinations,  Enforcement),  (3)  at  least  one  project  should  be 
considered  high  risk,  and  (4)  at  least  one  project  should  have  been 
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reviewed  by  Justice's  Information  Technology  Investment  Board  (ITIB). 

The  projects  we  evaluated  are: 

•  Coordinated  Interagency  Partnership  Regulating  International 
Students  (CIPRIS):  CIPRIS  is  an  Internet-based  system  that  is  intended 
to  modernize  and  streamline  the  current  process  for  collecting 
information  relating  to  nonimmigrant  foreign  students  and  other 
exchange  program  participants.  It  is  intended  to  enable  U.  S. 
universities,  schools,  and  cultural  exchange  programs  to  report  and 
share  information  electronically  with  INS  and  other  government 
regulatory  agencies.  INS  has  implemented  an  operational  prototype  of 
CIPRIS  at  21  educational  institutions.  CIPRIS  is  a  concept  exploration 
project  that  supports  the  Examinations  business  area  within  INS.  INS 
has  designated  CIPRIS  as  a  high-risk  project  and  it  has  been  reviewed  by 
Justice's  ITIB.  According  to  INS,  it  obligated  about  $3.1  million  for 
CIPRIS  in  fiscal  year  2000. 

•  Computer-Linked  Application  Information  Management  System 
(CLAIMS)  4.0:  According  to  INS,  CLAIMS  4  is  intended  to  improve 
delivery  of  naturalization  services  by  fully  automating  INS'  case 
management  system.  According  to  INS,  CLAIMS  4  supports  the 
Immigration  Services  Program  within  INS  and  is  currently  operational  at 
59  sites.  According  to  INS,  it  obligated  $14.5  million  for  CLAIMS  4  in 
fiscal  year  2000. 

•  Integrated  Surveillance  Intelligence  System  (ISIS):  ISIS  was 
established  to  detect  and  deter  illegal  intruders  and  to  safely  apprehend 
illegal  aliens  on  the  U.S.-Mexico  and  U.S.-Canada  borders.  ISIS  is 
designed  to  provide  all-weather  sensor  and  video  survefilance  of  the 
U.S.  borders  24  hours  a  day,  7  days  a  week.  The  major  components  of 
ISIS  are  the  Intelligent  Computer-Assisted  Detection  system,  groimd 
sensors,  and  the  Remote  Video  Surveillance  system.  ISIS  supports  the 
Enforcement  program  area  within  INS  and  has  been  reviewed  by 
Justice's  rriB.  According  to  INS,  it  obligated  about  $18  million  for  ISIS 
in  fiscal  year  2000  to  further  deploy  the  system. 

•  Central  Index  System  (CIS):  CIS  provides  INS  with  information  about 
persons  of  interest  to  the  INS.  According  to  INS,  CIS  also  interacts  with 
various  INS  databases  to  provide  the  data  necessary  for  INS  operations. 
CIS  currently  maintains  approximately  45  million  detailed  records  on 
individuals  of  interest  to  INS.  CIS  supports  the  INS'  Corporate  business 
area  and  is  in  the  operations  and  maintenance  phase  of  its  life  cycle. 
According  to  INS,  it  obligated  about  $2.6  million  for  CIS  in  fiscal  year 
2000. 
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We  did  not  validate  INS'  IT  spending  obligations  for  fiscal  year  2000  and  IT 
spending  estimates  for  fiscal  year  2001. 

To  supplement  our  document  reviews,  we  interviewed  senior  INS  officials, 
including  the  Deputy  Commissioner,  who  chairs  the  IRB,  and  the  Executive 
Associate  Commissioner  for  Management,  who  is  the  Chief  Information 
Officer  (CIO)  and  an  IRB  member.  We  also  interviewed  the  Acting 
Associate  Commissioner  for  Information  Resources  Management,  who 
chairs  the  ESC;  the  Director  of  INS'  Investment  Management  Team; 
portfolio  managers;  the  Director  of  the  Office  of  Strategic  Information  and 
Technology  Development;  IT  project  managers;  program  managers;  Office 
of  Budget  representatives;  and  officials  involved  with  the  development  and 
maintenance  of  INS'  asset  tracking  systems. 

We  compared  the  evidence  collected  from  our  document  review  and 
interviews  to  the  key  practices  and  critical  processes  in  ITIM.  Because 
ITIM  is  a  hierarchical  fi'amework,  the  rating  of  each  critical  process  is 
dependent  on  the  key  practices  below  it.  Therefore,  we  first  rated  the  key 
practices.  In  accordance  with  the  ITIM  assessment  method,  we  rated  a  key 
practice  as  “executed”  when  we  determined,  by  consensus,  that  INS  was 
executing  the  key  aspects  of  the  practice.  A  key  practice  was  rated  as  “not 
executed”  when  we  determined  that  there  were  significant  weaknesses  in 
INS'  execution  of  the  key  practice  and  INS  offered  no  adequate  alternative, 
or  when  the  team  foxmd  no  evidence  of  a  practice  during  the  review. 

Once  the  key  practices  were  rated,  we  rated  each  of  the  nine  critical 
processes  we  reviewed.  A  critical  process  was  rated  as  “implemented”  if  all 
of  the  xmderlying  key  practices  were  rated  as  being  executed.  A  critical 
process  was  rated  as  “not  implemented,  but  improvements  underway”  if 
over  half,  but  not  all,  of  its  underlying  key  practices  were  rated  as  being 
executed.  A  critical  process  was  rated  as  “not  implemented”  when  there 
were  significant  weaknesses  (i.e.,  fewer  than  50  percent  of  the  key 
practices  had  been  implemented)  in  INS'  implementation  of  the  underlying 
key  practices  and  no  adequate  alternative  was  in  place. 

To  determine  whether  the  Department  of  Justice  is  effectively  promoting, 
guiding,  and  overseeing  INS'  investment  management  activities,  we 
interviewed  officials  within  the  Office  of  Information  Management  and 
Security  Staff,  the  organization  that  plays  a  leading  role  in  Justice's 
investment  management  activities.  We  also  reviewed  Justice's  Januaiy  2000 
investment  management  guidance,  draft  policy  and  guidance  documents, 
INS  project  proposals,  ITIB  review  and  decision  documentation,  and 
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quarterly  briefing  documents.  We  also  discussed  Justice's  oversight 
activities  with  various  officials  within  INS. 

We  conducted  our  work  at  INS  and  Justice  headquarters  in  Washington, 
D.C.,  from  May  2000  through  October  2000  in  accordance  with  generally 
accepted  government  auditing  standards.  Justice's  Assistant  Attorney 
General  for  Administration  provided  written  comments  of  a  draft  of  this 
report.  These  comments  are  presented  in  chapter  6  and  are  reprinted  in 
appendix  1. 
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The  primary  purpose  of  ITIM  stage  two  maturity  is  to  attain  repeatable, 
successful  IT  project-level  investment  control  processes  and  basic 
selection  processes.  For  an  organization  to  develop  an  overall  sound  IT 
investment  management  process,  it  must  first  be  able  to  control  its 
investments  so  that  it  can  identify  expectation  gaps  early  and  correct  them. 
According  to  ITIIVI,  stage  two  maturity  includes  (1)  defining  IRB  operations, 

(2)  developing  a  basic  process  for  selecting  new  IT  proposals, 

(3)  developing  project-level  investment  control  processes,  (4)  creating  an 
IT  asset  inventory,  and  (5)  identifying  the  business  needs  for  each  IT 
project. 

INS  has  not  fully  implemented  any  of  the  critical  processes  associated  with 
stage  two;  however,  it  has  improvements  imderway  and  is  close  to  fully 
implementing  two  of  these  processes.  INS  has  (1)  established  an  IRB, 
which  comprises  both  IT  and  business  senior  executives  and  functions  as 
INS'  central  decision-making  body  for  IT  projects,  and  (2)  the  IRB  has 
followed  a  structured  process  for  developing  and  selecting  new  IT 
proposals  and  making  initial  funding  decisions  for  these  proposals. 

However,  INS  has  not  yet  developed  some  of  the  capabilities  necessary  to 
build  a  sound  IT  investment  management  process.  For  example,  INS  has 
not  (1)  established  basic  project-level  control  processes  to  ensure  that  its 
IT  projects  are  performing  as  expected,  (2)  created  an  IT  asset  inventory 
for  investment  management,  and  (3)  defined  business  needs  for  all  of  its  IT 
projects.  According  to  INS,  it  lacks  these  critical  investment  capabilities 
because  it  has  not  yet  made  IT  investment  management  an  institutional 
priority.  Table  1  summarizes  INS'  stage  two  maturity. 
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Table  1 :  Summary  of  Stage  IVvo  Critical  Process  Ratings 


Critical  process 

Rating 

Key  practices 

Key  practices 
executed 

IT  Investment  Board 
Operation 

Not  Implemented,  but 
improvements  underway 

6 

4 

IT  Project  Oversight 

Not  implemented 

11 

2 

IT  Asset  Tracking 

Not  implemented 

8 

2 

Business  Needs 
Identification  for  IT 
Projects 

Not  implemented 

8 

4 

Proposal  Selection 

Not  implemented,  but 
Improvements  underway 

6 

5 

Total 

39 

17 

INS'  capabilities  for  each  of  the  stage  two  critical  processes  are  discussed 
below. 
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INS  Has  Established  an 
IRB,  But  Has  Not 
Developed  Policies  and 
Procedures  to  Govern 
IRB  Operations 


The  purpose  of  this  critical  process  is  to  define  and  establish  the  governing 
board  or  boards  responsible  for  selecting,  controlling,  and  evaluating  IT 
investments.  This  includes  defining  the  membership,  guiding  policies, 
operations,  roles  and  responsibilities,  and  authorities  for  the  investment 
board  and,  if  appropriate,  each  board's  support  staff.  These  policies,  roles 
and  responsibilities,  and  authorities  also  provide  the  basis  for  the  board's 
investment  selection,  control,  and  evaluation  activities. 


According  to  ITIM,  effective  IT  investment  board  operations  require, 
among  other  things,  that  (1)  the  board  membership  include  both  FT  and 
business  knowledge,  (2)  the  organization's  executives  and  line  managers 
support  and  carry  out  board  decisions,  (3)  the  organization  create  an 
organization-specific  process  guide  that  includes  policies  and  procedures 
to  direct  the  board's  operations,  and  (4)  the  IRB  operate  according  to  these 
written  policies  and  procedures. 

INS  is  executing  many  of  the  practices  in  this  critical  process.  For  example, 
INS  has  an  IRB  that  functions  as  a  central  decision-making  body  for  IT 
investments  and  is  composed  of  senior  executives  fi'om  both  INS'  IT  and 
business  areas.  During  our  discussions  with  agency  officials,  we  foxmd 
broad  support  within  the  organization  for  the  IRB's  decisions.  For  example, 
three  of  the  four  program/project  managers  we  interviewed  acknowledged 
the  IRB's  role  in  investment  decision-making.  The  IRB  is  chaired  by  the 
Deputy  Commissioner  and  includes  INS'  three  Executive  Associate 
Commissioners.  The  IRB  is  supported  by  an  ESC,  which  is  comprised  of 
senior  representatives  who  manage  INS'  eight  IT  portfolios.  The  ESC 
reviews  and  analyzes  IT  investments  and  makes  recommendations  to  the 
IRB  for  final  approval.  This  senior  level  involvement  and  the  breadth  of 
representation  help  to  demonstrate  executive  sponsorship  of  the  process 
and  support  for  the  projects  selected. 

While  INS  has  an  IRB,  it  is  not  functioning  according  to  written  policies  and 
procedures.  Instead,  the  IRB  operates  according  to  imdocumented 
procedures  for  selecting  new  IT  proposals.  According  to  the  Director  of 
INS'  Investment  Management  Team,  INS  has  begun  developing  written 
policies  and  procedures  and  plans  to  complete  them  about  March  2001. 
However,  until  INS  develops  and  implements  these  policies  and 
procedmes,  key  IT  investment  activities  may  not  be  done  consistently,  if  at 
aU.  Table  2  summarizes  the  ratings  for  each  key  practice  and  the  specific 
findings  supporting  the  ratings. 
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Table  2:  Summary  of  Ratings  and  Evidence  for  the  IT  Investment  Board  Operation  Critical  Process 

Key  practice 

Rating 

Summary  of  evidence 

Organizational 

commitment 

1 .  An  organization-specific  IT 
investment  process  guide  is  created 
to  direct  each  board's  operations. 

Not  executed 

INS  does  not  have  an  organization-specific  IT  Investment 
process  guide  to  direct  the  board's  operations.  According 
to  the  Director  of  INS'  Investment  Management  Team, 

INS  Is  developing  a  process  guide  and  plans  to  complete 
it  by  March  2001 . 

2.  Organization  executives  and  line 
managers  support  and  carry  out  IT 
investment  board  decisions. 

Executed 

INS  executives  and  line  managers  support  the  IRB's 
decisions.  Three  of  the  four  program/project  managers 
we  interviewed  acknowledged  the  IRB's  role  in 
investment  decision-making. 

Prerequisites 

1 .  Adequate  resources  are  provided 
for  operating  each  IT  investment 
board. 

Executed 

INS  has  adequate  resources  for  operating  the  Investment 
board.  Resources  include  both  internal  staff  support  and 
contractor-provided  support.  Also,  the  IRB  has  been 
operating  as  a  decisional  body  since  the  Fall  of  1998. 

2.  Board  members  understand  the 
investment  board's  policies  and 
procedures  and  exhibit  core 
competencies  in  using  the  IT 
investment  approach  via  training, 
education,  or  experience. 

Executed 

IRB  members  understand  the  IRB's  Informal  practices  for 
investing  in  IT  and  exhibit  competencies  in  using  the  IT 
investment  approach. 

Activities 

1 .  Each  investment  board  is  created 
and  defined  with  board  membership 
integrating  both  IT  and  business 
knowledge. 

Executed 

IRB  membership  includes  representatives  from  both  IT 
and  business  areas  within  INS. 

2.  Each  IT  Investment  board 
operates  according  to  written 
policies  and  procedures  in  the 
organization-specific  IT  investment 
process  guide. 

Not  executed 

The  IRB  does  not  operate  according  to  written 
procedures.  INS  does  not  have  an  organization-specific 

IT  investment  process  guide  (See  organizational 
commitment  1).  However,  the  IRB  operates  according  to 
undocumented,  established  procedures  for  selecting  new 

IT  proposals. 
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The  purpose  of  project  oversight  is  to  ensure  that  the  IRB  provides 
effective  oversight  for  its  ongoing  IT  projects  throughout  all  phases  of  their 
life  cycle.  Under  stage  2  maturity,  the  IRB  should  review  each  project's 
progress  toward  predefined  cost  and  schedule  expectations,  using 
established  criteria,  and  take  corrective  actions  when  cost  estimates  and 
project  milestones  are  not  achieved.  Implementing  this  critical  process 
provides  the  basis  for  evolving  the  organization's  IT  investment  control 
activities. 

According  to  ITIM,  effective  project  oversight  requires,  among  other  things, 

(1)  having  written  polices  and  procedures  for  project  management, 

(2)  developing  and  maintaining  an  approved  project  management  plan  for 
each  IT  project,  (3)  having  written  policies  and  procedures  for  oversight  of 
IT  projects,  (4)  making  up-to-date  cost  and  schedule  data  for  each  project 
available  to  the  IRB,  (6)  reviewing  each  project's  performance  by 
comparing  actual  cost  and  schedule  data  to  expectations  regularly,  and 
(6)  ensuring  that  corrective  actions  for  each  underperforming  project  are 
defined,  implemented,  and  tracked  until  the  desired  outcome  is  achieved. 

INS  is  not  effectively  overseeing  its  IT  projects.  While  INS  has  documented 
policies  and  procedures  for  project  management  in  its  System 
Development  Life  Cycle  (SDLC)  manual,  it  is  not  following  its  own 
procedures.  For  example,  INS  has  not  developed  and  maintained  project 
management  plans  that  include  cost  and  schedule  controls  for  each  of  its  IT 
projects,  an  SDLC  requirement.  In  fact,  only  two  of  the  four  projects  that 
we  reviewed  had  current  project  management  plans. 

Furthermore,  INS  does  not  have  written  polices  and  procedures  for 
oversight  of  its  IT  projects.  Without  written  polices  and  procediues,  INS 
increases  the  risk  that  project  oversight  activities  will  not  be  performed 
effectively.  For  example,  the  IRB  does  not  (1)  receive  up-to-date  cost  and 
schedule  data  for  each  project,  (2)  oversee  each  project's  performance 
regularly  by  comparing  actual  cost  and  schedule  data  to  expectations,  and 

(3)  ensure  that  corrective  actions  are  implemented  and  tracked  for 
underperforming  projects.  In  the  absence  of  effective  oversight,  INS 
executives  do  not  have  adequate  assiuance  that  IT  projects  are  being 
developed  on  schedule  and  within  budget.  Table  3  summarizes  the  ratings 
for  each  key  practice  and  the  specific  findings  supporting  the  ratings. 


INS  Is  Not  Effectively 
Overseeing  Its  Ongoing 
IT  Projects 
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Table  3:  Summary  of  Ratings  and  Evidence  for  the  IT  Project  Oversight  Critical  Process 

Key  practice 

Rating 

Summary  of  evidence 

Organizational 

commitment 

1 .  The  organization  has  written 
policies  and  procedures  for  project 
management. 

Executed 

INS'  SDLC  manual  contains  policies  and  procedures  for 
project  management. 

2.  The  organization  has  written 
policies  and  procedures  for 
management  oversight  of  IT 
projects. 

Not  executed 

INS  indicated  In  its  self-assessment  that  this  key  practice 
was  “not  executed." 

Prerequisites 

1 .  Adequate  resources  are  provided 
to  assist  the  board(s)  In  overseeing 
IT  projects. 

Not  executed 

According  to  INS,  it  does  not  have  adequate  resources  to 
assist  the  board  in  overseeing  IT  projects. 

2.  Each  IT  project  has  and 
maintains  an  approved  project 
management  plan  that  includes  cost 
and  schedule  controls. 

Not  executed 

According  to  INS'  officials,  not  all  projects  have  project 
management  plans.  Two  of  the  four  case  study  projects 
we  reviewed  did  not  have  project  management  plans. 

3.  An  IT  investment  board  is 
operating. 

Executed 

The  IRB  is  functioning  as  the  central  decision-making 
body  for  IT  projects,  although  it  is  not  operating 
according  to  written  policies  and  procedures  (See  IT 
Investment  Board  Operation:  organizational  commitment 

1). 

4.  Information  from  the  IT  asset 
inventory  is  used  by  the  IT 
Investment  board  as  applicable. 

Not  executed 

The  IRB  does  not  use  information  from  an  IT  asset 
inventory. 

Activities 

1 .  Each  project's  up-to-date  cost 
and  schedule  data  are  provided  to 
the  appropriate  IT  investment  board. 

Not  executed 

Up-to-date  cost  and  schedule  data  are  not  provided  to 
the  IRB  for  each  project.  Three  of  the  four  projects  we 
reviewed  did  not  provide  up-to-date  cost  and  schedule 
data  to  the  IRB. 

2.  Using  established  criteria,  the  IT 
Investment  board  oversees  each  IT 
project's  performance  regularly  by 
comparing  actual  cost  and  schedule 
data  to  expectations. 

Not  executed 

INS  indicated  In  Its  self-assessment  that  this  key  practice 
was  “not  executed.” 

3.  The  IT  investment  board  performs 
special  reviews  of  projects  that  have 
not  met  predetermined  performance 
standards. 

Not  executed 

INS  indicated  in  its  self-assessment  that  this  key  practice 
was  “not  executed.” 

4.  Appropriate  corrective  actions  for 
each  under  performing  project  are 
defined,  documented,  and  agreed  to 
by  the  IT  investment  board  and  the 
project  manager. 

Not  executed 

INS  indicated  in  its  self-assessment  that  this  key  practice 
was  “not  executed.” 

5.  Corrective  actions  are 
Implemented  and  tracked  until  the 
desired  outcome  is  achieved. 

Not  executed 

INS  indicated  in  its  self-assessment  that  this  key  practice 
was  “not  executed.” 
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The  purpose  of  the  asset  tracking  critical  process  is  to  create  and  maintain 
an  IT  asset  inventory  to  assist  in  managerial  decision-making.  To  make 
good  investment  decisions,  an  organization  must  know  where  its  IT  assets 
(i.e.,  personnel,  systems,  applications,  hardware,  software  licenses,  etc.) 
are  located  and  how  funds  are  being  expended  toward  acquiring, 
maintaining,  and  deploying  them.  This  critical  process  identifies  IT  assets 
within  the  organization  and  creates  a  comprehensive  inventory  of  them. 
This  inventory  can  take  many  forms,  but  regardless  of  form,  the  inventory 
should  identify  each  asset  and  its  associated  components. 

Beyond  identifying  IT  assets,  this  process  is  used  to  support  other  ITIM 
critical  processes  by  serving  as  an  investment  information  and  data 
repository  that  contains  such  items  as  the  list  of  systems  and  projects  and 
data  on  each  project's  progress  toward  achieving  its  plans.  To  support 
investment  decision-making,  this  inventory  shoirld  also  be  accessible 
where  it  is  of  the  most  value  to  decisiorunakers. 

According  to  ITIM,  effectively  tracking  IT  assets  requires,  among  other 
things,  (1)  making  investment  information  available  on  demand  to 
decisionmakers,  (2)  developing  and  maintaining  an  IT  asset  inventory 
according  to  written  procedures,  (3)  overseeing  the  development  and 
maintenance  of  the  asset  tracking  process,  and  (4)  assigning  responsibility 
for  managing  this  tracking  process. 

INS  has  not  implemented  an  effective  IT  asset  tracking  process  for 
investment  management.  While  investment  information  from  various 
sources  has  been  available  to  the  IRB  on  an  ad  hoc  basis,  it  is  not  available 
on  demand  and  INS  has  not  developed  and  maintained  an  inventory  for 
investment  management  purposes  according  to  written  policies  and 
procedures.  In  addition,  the  IRB  does  not  oversee  IT  asset  tracking 
activities  and  has  not  assigned  responsibility  for  managing  this  tracking 
process  to  support  investment  decision-making.  In  the  absence  of  standard, 
documented  procedures  for  developing  and  maintaining  the  inventory,  INS 
executives  do  not  have  adequate  assurance  that  timely,  complete,  and 
consistent  asset  data  are  available  to  them.  Table  4  summarizes  the  ratings 
for  each  key  practice  and  the  specific  findings  supporting  the  ratings. 


INS  Is  Not  Tracking 
and  Using  IT  Asset 
Information  for 
Investment 

Management  Purposes 
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Table  4:  Summary  of  Ratings  and  Evidence  for  the  IT  Asset  Tracking  Critical  Process 


IT  Asset  Tracking 

Rating 

Summary  of  evidence 

Organizational 

commitment 

1 .  The  organization  has  written 
policies  and  procedures  for 
developing  and  maintaining  an  IT 
asset  inventory. 

Not  executed 

INS  does  not  have  written  policies  and  procedures  for 
developing  and  maintaining  an  IT  asset  inventory  to 
support  investment  management. 

2.  An  official  is  assigned  responsibility 
for  managing  the  IT  asset  tracking 
process. 

Not  executed 

INS  has  not  assigned  responsibility  for  managing  the  IT 
asset  tracking  process  to  support  IT  investment  decision¬ 
making. 

Prerequisites 

1.  Adequate  resources  are  provided 
for  performing  the  IT  asset  tracking 
activities. 

Executed 

According  to  INS,  it  has  adequate  resources  for 
performing  IT  asset-tracking  activities. 

2.  An  IT  investment  board  exists  and 
oversees  the  development  and 
maintenance  of  IT  asset  tracking 
activities. 

Not  executed 

The  IRB  does  not  oversee  the  development  and 
maintenance  of  IT  asset-tracking  activities. 

Activities 

1 .  The  organization's  IT  asset 
inventory  is  developed  and 
maintained  according  to  a  written 
procedure. 

Not  executed 

INS  has  not  developed  an  IT  asset  inventory  to  support  IT 
investment  decision-making. 

2.  IT  asset  inventory  changes  are 
maintained  according  to  a  written 
procedure. 

Not  executed 

INS  has  not  developed  an  IT  asset  inventory  to  support  IT 
investment  decision-making. 

3.  Investment  information  is  available 
on  demand  to  decisionmakers  and 
other  affected  parties. 

Executed 

Investment  information  is  available  to  decisionmakers  on 
an  ad  hoc  basis  from  various  repositories. 

4.  Historical  IT  asset  inventory 
records  are  maintained  for  future 
selections  and  assessments. 

Not  executed 

INS  has  not  developed  an  IT  asset  inventory  to  support  IT 
investment  decision-making. 

Page  39 


GAO-01-146  INS’  IT  Investments 


Chapter  2 

INS  Lacks  Foundation  Capabilities  Upon 
Which  to  Build  IT  Investment  Management 
Maturity 


INS  Has  Not  Defined 
Business  Needs  for  All 
Its  IT  Projects 


The  pxirpose  of  defining  business  needs  for  each  IT  project  is  to  ensure  that 
each  project  supports  the  organization's  business  needs  and  meets  users' 
needs.  Thus,  this  critical  process  creates  the  link  between  the 
organization's  business  objectives  and  its  IT  management  strategy. 
According  to  ITIM,  effectively  identifying  business  needs  requires,  among 
other  things,  (1)  defining  the  organization's  business  needs  or  stated 
mission  goals,  (2)  identifying  users  for  each  project  who  will  participate  in 
the  project ‘s  development  and  implementation,  (3)  defining  business 
needs  for  each  project,  and  (4)  training  IT  staff  in  business  needs 
identification. 


INS  has  executed  some  of  the  key  practices  associated  with  effectively 
defining  business  needs  for  IT  projects.  For  example,  INS  has  (1)  defined 
its  business  needs  and  mission  goals  in  its  annual  performance  plan  and 
(2)  identified  users  for  its  projects  who  participate  in  the  project ‘s 
development  and  implementation.  However,  INS  has  not  clearly  defined 
specific  business  needs  for  each  project.  In  addition,  only  one  of  the  four 
project  managers  that  we  interviewed  stated  that  he  or  she  had  been 
trained  in  business  needs  identification.  In  the  absence  of  documented 
business  needs,  the  IRB  cannot  ensure  that  it  is  selecting  IT  investments 
that  meet  its  mission  needs  and  priorities.  Table  5  summarizes  the  ratings 
for  each  key  practice  and  the  specific  findings  supporting  the  ratings. 
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Table  5:  Summary  of  Ratings  and  Evidence  for  the  Business  Needs  Identification  for  IT  Projects  Critical  Process 

Key  practice 

Rating 

Summary  of  evidence 

Organizational 

commitment 

1.  The  organization  has  written 
policies  and  procedures  for  identifying 
the  business  needs  (and  the 
associated  users)  of  each  IT  project. 

Executed 

The  SDLC  manual  and  a  requirements  management 
process  guide  contain  written  policies  and  procedures 
for  identifying  the  business  needs  and  associated  users 
of  each  IT  project.  The  Technical  Bulletin  for  Chartering 
User  Groups  for  Automation  Projects  6ei\nes  procedures 
for  identifying  the  associated  users  of  IT  projects. 

However,  three  of  the  four  program  managers  whom  we 
interviewed  were  not  aware  of  these  policies  and 
procedures. 

Prerequisites 

1.  Adequate  resources  are  provided 
for  identifying  business  needs  and 
associated  users. 

Not  executed 

According  to  INS,  it  does  not  have  adequate  resources 
to  identify  business  needs  and  associated  users.  While 
the  Office  of  Strategic  Information  and  Technology 
Development  has  been  assigned  responsibility  for  doing 
this,  it  is  not  fully  staffed. 

2.  The  organization  has  defined 
business  needs  or  stated  mission 
goals. 

Executed 

INS'  mission  goals  are  defined  in  its  annual  performance 
plan. 

3.  IT  staff  are  trained  in  business 
needs  identification. 

Not  executed 

IT  staff  are  not  consistently  trained  in  business  needs 
identification.  Only  one  of  the  four  IT  project  managers 
whom  we  interviewed  stated  that  he  or  she  was  trained 
in  business  needs  identification. 

4.  All  IT  projects  are  identified  In  the  IT 
asset  inventory. 

Not  executed 

INS  has  a  list  of  IT  projects.  However,  it  is  not  maintained 
as  part  an  IT  asset  inventory. 

Activities 

1 .  The  business  needs  for  each  IT 
project  are  clearly  identified  and 
defined. 

Not  executed 

According  to  INS  officials,  business  needs  for  each  IT 
project  are  not  always  identified  and  identified  business 
needs  are  not  always  clear.  Two  of  the  four  case  study 
projects  we  reviewed  did  not  have  identified  business 
needs. 

2.  Specific  users  are  identified  for 
each  IT  project. 

Executed 

The  four  case  study  projects  we  reviewed  did  have 
identified  users. 

3.  Identified  users  participate  in 
project  management  throughout  a 
project's  life  cycle. 

Executed 

For  the  four  case  study  projects  we  reviewed,  identified 
users  do  participate  in  project  management  throughout 
the  life  cycle. 
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INS  Has  a  Structured 
Process  for  Selecting 
New  IT  Proposals  But 
Has  Not  Consistently 
Analyzed  Them 
According  to 
Established  Criteria 


The  purpose  of  proposal  selection  is  to  establish  a  structured  process  for 
selecting  new  IT  proposals.  According  to  ITIM,  effective  proposal  selection 
requires,  among  other  things,  (1)  designating  an  official  to  manage  the 
proposal  selection  process,  (2)  using  a  structured  process  to  develop  new 
IT  proposals,  (3)  making  funding  decisions  for  new  IT  proposals  according 
to  an  established  selection  process,  and  (4)  analyzing  and  ranking  new  IT 
proposals  according  to  established  selection  criteria,  including  cost  and 
schedule  criteria. 

INS  has  established  a  structured  process  for  selecting  new  IT  proposals. 
The  Deputy  Commissioner,  as  the  Chair  of  the  IRB,  is  designated  to  manage 
INS'  proposal  selection  process.  In  addition,  INS  uses  a  structured  process 
to  develop  new  proposals  and  makes  initial  funding  decisions  for  these 
proposals.  However,  INS  has  not  consistently  analyzed  and  ranked  these 
proposals  according  to  established  selection  criteria.  Established  selection 
criteria  would  assist  IT  managers  in  creating  proposals  that  best  meet  the 
needs  and  priorities  of  INS.  Table  6  summarizes  the  ratings  for  each  key 
practice  and  the  specific  findings  supporting  the  ratings. 
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Table  6:  Summary  of  Ratings  and  Evidence  for  the  Proposal  Selection  Critical  Process 


Key  practice 

Rating 

Summary  of  evidence 

Organizational 

commitment 

1 .  Executives  and  managers  follow 
an  established  selection  process. 

Executed 

INS'  IRB  follows  an  established  process  for  submitting 
and  selecting  IT  proposals. 

2.  An  official  is  designated  to 
manage  the  proposal  selection 
process. 

Executed 

INS'  Deputy  Commissioner,  as  Chair  of  the  IRB,  is 
designated  to  manage  the  proposal  selection  process. 

Prerequisites 

1 .  Adequate  resources  are  provided 
for  proposal  selection  activities. 

Executed 

According  to  INS,  it  has  adequate  resources  for  proposal 
selection  activities.  These  resources  Include  the  ESC 
members,  IRB  support  staff,  and  contractor  support. 

Activities 

1 .  The  organization  uses  a 
structured  process  to  develop  new 

IT  proposals. 

Executed 

INS  uses  a  structured  process  to  develop  new  IT 
proposals.  This  process  involves  submitting  new 
proposal  templates  (IRB  presentation  packages)  to 
portfolio  managers  who  evaluate  them  and  forward  them 
to  the  ESC.  The  ESC  recommends  proposals  to  the  IRB 
for  review  and  approval. 

2.  Executives  analyze  and  prioritize 
new  IT  proposals  according  to 
established  selection  criteria. 

Not  executed 

Executives  do  not  consistently  analyze  and  rank  new  IT 
proposals  according  to  established  selection  criteria. 

3.  Executives  make  funding 
decisions  for  new  IT  proposals 
according  to  an  established 
process. 

Executed 

An  established  process  has  been  used  to  make  funding 
decisions. 
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An  IT  investment  portfolio  is  a  collection  of  investments  that  are  assessed 
and  managed  based  on  common  criteria.  While  an  organization  may  have 
more  than  one  level  of  investment  portfolios,  it  should  always  have  an 
enteiprisewide  portfolio.  Managing  investments  as  a  portfolio  is  a 
conscious,  continuous,  and  proactive  approach  to  expending  limited 
resources  on  aU  competing  initiatives  in  light  of  the  relative  beneficial 
effects  of  these  investments.  Taking  an  enterprisewide  portfolio 
perspective  enables  an  organization  to  consider  its  investments 
comprehensively  so  that  the  investments  address  its  mission,  strategic 
goals,  and  objectives.  A  portfolio  approach  also  allows  an  organization  to 
determine  priorities  and  make  decisions  about  which  projects  to  fund 
based  on  analyses  of  the  relative  costs,  benefits,  and  risks  of  all  projects, 
including  projects  that  are  proposed,  imder  development,  and  in  operation. 

The  purpose  of  ITM  stage  three  maturity  is  to  create  and  manage  IT 
investments  as  a  complete  enterprise  investment  portfolio.  Once  ongoing 
projects  can  be  implemented  on  schedule  and  within  budget  as  is 
emphasized  in  stage  two,  the  organization  is  capable  of  managing  its 
projects  as  an  investment  portfolio.  According  to  FTIM,  stage  three  maturity 
includes  (1)  defining  portfolio  selection  criteria,  (2)  engaging  in  project- 
level  investment  analysis,  (3)  developing  a  complete  portfolio  based  on  the 
investment  analysis,  and  (4)  maintaining  oversight  over  the  investment 
performance  of  the  portfolio. 

INS  has  not  implemented  any  of  the  critical  processes  in  stage  three.  In 
general,  INS  has  not  created  the  associated  policies  and  procedures  to 
initiate  or  perpetuate  any  of  the  critical  processes,  and  as  a  result,  it  has  not 
systematically  collected  and  analyzed  the  data  needed  to  make  soimd  and 
informed  decisions  about  competing  investment  choices,  which 
consciously  consider  value  and  risk.  In  addition,  while  INS  has  established 
eight  portfolio  categories,  it  has  not  established  an  enterprisewide 
investment  portfolio.  Therefore,  decisions  may  be  made  between 
competing  investments  within  a  business  area,  but  INS  cannot  make  trade¬ 
offs  between  investments  across  the  enterprise  to  determine  which 
projects  contribute  most  to  the  agency  mission  and  priorities.  According  to 
INS  officials,  INS  has  not  yet  made  IT  investment  management  an 
institutional  priority.  Table  7  summarizes  INS'  stage  tlu  o  maturity. 
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Table  7:  Summary  of  Stage  Three  Critical  Process  Ratings 


Critical  process 

Rating 

Key  practices 

Key  practices 
executed 

Portfolio  Selection 
Criteria  Definition 

Not  implemented 

6 

1 

Investment  Analysis 

Not  implemented 

7 

1 

Portfolio  Development 

Not  implemented 

9 

4 

Portfolio  Performance 
Oversight 

Not  implemented 

9 

0 

Total 

31 

6 

INS'  capabilities  for  each  of  the  stage  three  critical  processes  are  discussed 
below. 
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INS  Has  Not  Created 
Useful  Portfolio 
Selection  Criteria 


Portfolio  selection  criteria  make  up  a  necessary  part  of  an  IT  investment 
management  process.  Developing  an  enterprisewide  investment  portfolio 
involves  defining  appropriate  investment  cost,  benefit,  schedule,  and  risk 
criteria  to  ensure  that  the  selected  investments  will  best  support  the 
organization's  strategic  goals,  objectives,  and  mission.  Thus,  portfolio 
selection  criteria  need  to  reflect  the  enterprisewide  and  strategic  focus  of 
the  oi^anization.  In  addition,  the  criteria  should  (1)  include  cost,  benefit, 
schedule,  and  risk  elements,  which  serve  to  create  a  common  set  of  criteria 
that  are  used  to  compare  projects  of  different  types  to  one  another  and 
(2)  be  clearly  communicated  to  project  managers  throughout  the 
organization  so  that  these  managers  can  take  the  criteria  into  accoimt 
when  developing  proposals.  Without  portfolio  selection  criteria,  projects 
may  be  selected  on  the  basis  of  isolated  business  needs,  the  type  and 
availability  of  funds,  or  the  receptivity  of  management  to  a  specific  project 
proposal. 


Thus,  according  to  ITIM,  developing  portfolio  selection  criteria  requires, 
among  other  things,  that  (1)  an  investment  board  approve  the  criteria, 
including  cost,  benefit,  schedule,  and  risk  criteria;  (2)  the  criteria  be 
distributed  throughout  the  organization;  (3)  adequate  resources  be 
provided  for  selection  criteria  defmition  activities;  and  (4)  a  working  group 
be  responsible  for  creating  and  modifying  the  criteria. 

INS  developed  criteria  for  selecting  new  proposals;  however,  the  criteria 
had  not  been  approved  by  the  IRB  and  did  not  consistently  include  cost, 
schedule,  benefit,  and  risk  criteria.  Furthermore,  INS  had  not  distributed 
the  criteria  throughout  INS.  For  example,  none  of  the  IT  project  and 
program  managers  that  we  interviewed  were  aware  of  the  selection  criteria 
that  had  been  developed.  In  addition,  while  INS  indicated  that  it  has 
adequate  resources  to  develop  complete  portfolio  selection  criteria,  it  has 
not  designated  a  working  group  to  create  and  modify  the  criteria.  Without 
useful  selection  criteria,  INS  is  missing  a  critical  means  of  ensuring  that 
selected  investments  best  support  the  organization's  mission  and  priorities. 
Table  8  summarizes  the  ratings  for  each  key  practice  and  the  specific 
findings  supporting  the  ratings. 
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INS  Is  Not  Managing  Its  IT  Investments  as  a 

Complete  Portfolio 

Table  8:  Summary  of  Ratings  and  Evidence  for  the  Portfolio  Selection  Criteria  Definition  Critical  Process 

Key  practice 

Rating 

Summary  of  evidence 

Organizational 

commitment 

1 .  The  organization  has  written 
policies  and  procedures  for  creating 
and  modifying  IT  portfolio  selection 
criteria. 

Not  executed 

INS  does  not  have  written  policies  and  procedures  for 
creating  and  modifying  IT  portfolio  selection  criteria. 

Prerequisites 

1 .  Adequate  resources  are  provided 
for  selection  criteria  definition 
activities. 

Executed 

According  to  INS,  it  has  adequate  resources  and  staff  for 
selection  criteria  definition  activities.  These  resources 
include  the  ESC  members. 

2.  A  working  group  is  designated  to 
be  responsible  for  creating  and 
modifying  the  IT  portfolio  selection 
criteria. 

Not  executed 

Responsibility  for  creating  and  modifying  IT  portfolio 
selection  criteria  is  not  designated. 

Activities 

1 .  The  enterprisewide  IT  investment 
board  approves  the  core  IT  portfolio 
selection  criteria,  including  cost, 
benefit,  schedule,  and  risk  (CBSR) 
criteria,  based  on  the  organization's 
mission,  goals,  strategies,  and 
priorities. 

Not  executed 

INS  has  not  developed  selection  criteria  that  consistently 
included  CBSR  criteria  and  that  have  been  applied  to  all 

IT  investments,  including  ongoing  investments. 

2.  The  IT  portfolio  selection  criteria 
are  distributed  throughout  the 
organization. 

Not  executed 

Selection  criteria  have  not  been  distributed  throughout 
the  organization.  None  of  the  four  project  managers  who 
we  interviewed  were  aware  of  criteria  used  by  the  IRB  to 
select  IT  investments. 

3.  The  IT  portfolio  selection  criteria 
are  reviewed  using  cumulative 
experience  and  event-driven  data 
and  modified,  as  appropriate. 

Not  executed 

INS  has  reviewed  its  IT  portfolio  selection  criteria. 
However,  the  criteria  have  not  been  developed  for  all 
investments  (See  activity  1). 
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INS  Is  Not  Managing  Its  IT  Investments  as  a 
Complete  Portfolio 


INS  Does  Not  Analyze 
Its  IT  Investments 
Based  on  Cost,  Benefit, 
Schedule,  and  Risk 
Data  When  Making 
Investment  Decisions 


The  purpose  of  investment  analysis  is  to  ensure  that  all  IT  investments  are 
consistently  analyzed  and  prioritized  according  to  the  organization's 
portfolio  selection  criteria,  which  should  include  cost,  benefit,  schedule, 
and  risk  criteria.  According  to  ITIM,  effective  investment  analysis  includes, 
among  other  things,  that  (1)  portfolio  selection  criteria  have  been 
developed;  (2)  the  IRB  ensures  that  cost,  benefit,  schedule,  and  risk  data 
are  assessed  and  validated  for  each  investment;  (3)  the  IRB  compares  each 
investment  against  the  organization's  portfolio  selection  criteria;  and 
(4)  the  IRB  creates  a  ranked  list  of  investments  using  the  portfolio  selection 
criteria. 


INS'  IRB  does  not  analyze  and  rank  proposed  and  ongoing  investments 
based  on  their  expected  cost,  benefit,  schedule,  and  risk.  As  mentioned 
previously,  INS  has  not  developed  selection  criteria  that  include  these 
elements,  nor  has  it  ensimed  that  cost,  benefit,  schedule,  and  risk  data  are 
assessed  and  validated  for  each  IT  investment.  For  example,  none  of  the 
four  projects  we  reviewed  provided  cost,  benefit,  schedule,  or  risk  data  to 
INS'  IRB  for  consideration  during  the  selection  process.  Instead,  the  IRB 
focused  on  the  near-term  cost  (e.g.,  annual  budget  dollars)  of  each  project 
and  the  perceived  importance  of  the  project  to  INS'  mission.  In  the  absence 
of  portfolio  selection  criteria  and  good  investment-related  data  (i.e.,  cost, 
benefit,  schedule,  and  risk  data),  the  IRB  cannot  compare  and  analyze  its 
investments  based  on  their  cost,  benefit,  schedule,  and  risk  expectations 
and  create  a  ranked  list  of  investments  that  best  align  with  mission 
improvement  goals  and  organizational  direction.  As  a  result,  INS  is  missing 
critical  information  for  making  sovmd  IT  investment  decisions.  Table  9 
summarizes  the  ratings  for  each  key  practice  and  the  specific  findings 
supporting  the  ratings. 
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INS  Is  Not  Managing  Its  IT  Investments  as  a 
Complete  Portfolio 


Table  9:  Summary  of  Ratings  and  Evidence  for  the  Investment  Analysis  Critical  Process 


Key  practice 

Organizational 

commitment 


1 .  The  organization  has  written 
policies  and  procedures  for 
analyzing  IT  Investments. 


Rating  Summary  of  evidence 

Not  executed  INS  does  not  have  written  policies  and  procedures  for 
analyzing  IT  investments. 


Prerequisites 

1 .  Adequate  resources  are  provided 
for  Investment  analysis  activities. 

Executed 

According  to  INS,  it  has  adequate  resources  for 

Investment  analysis  activities.  These  resources  include 
the  ESC  members. 

2.  IT  investment  portfolio  selection 
criteria  have  been  developed. 

Not  executed 

IT  investment  portfolio  selection  criteria  have  not  been 
developed  for  all  investments  (See  the  Portfolio  Selection 

Criteria  Definition  critical  process). 

3.  Information  from  the  IT  asset 
inventory  is  used  by  the  IT 
investment  board. 

Not  executed 

The  IRB  does  not  use  information  from  an  IT  asset 
inventory. 

Activities 

1 .  Each  IT  Investment  board 
ensures  that  the  CBSR  data  and 
other  required  data  are  validated  for 
each  investment  within  its  span  of 
control. 

Not  executed 

The  IRB  does  not  ensure  that  cost,  benefit,  schedule,  or 
risk  data  are  validated.  None  of  the  four  project 
managers  that  we  Interviewed  provided  this  data  to  the 

IRB. 

2.  Each  IT  Investment  board 
assesses  each  of  its  IT  investments 
with  respect  to  the  IT  portfolio 
selection  criteria. 

Not  executed 

The  IRB  does  not  assess  each  of  Its  IT  Investments  with 
respect  to  IT  portfolio  selection  criteria  (See  prerequisite 

2). 

3.  Each  IT  investment  board 
prioritizes  its  full  portfolio  of  IT 
Investments  using  the  portfolio 
selection  criteria. 

Not  executed 

The  IRB  does  not  prioritize  its  full  portfolio  of  IT 
investments  using  portfolio  selection  criteria  (See 
prerequisite  2). 
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INS  Is  Not  Managing  Its  IT  Investments  as  a 
Complete  Portfolio 


INS  Does  Not 
Comparatively  Assess 
All  Its  IT  Projects 
When  Making 
Selections  for  Funding 


The  purpose  of  the  portfolio  development  process  is  to  ensure  that  the  IRB 
analyzes  and  compares  all  IT  investments  to  select  and  fund  those  with 
manageable  risks  and  returns  and  that  best  address  the  strategic  business 
direction  and  priorities  of  the  organization.  Once  this  is  accomplished, 
investments  can  be  compared  to  one  another  within  and  across  the 
portfolio  categories  and  the  best  overall  portfolio  can  then  be  selected  for 
funding. 


According  to  ITIM,  portfolio  development  requires,  among  other  things, 

(1)  defining  common  portfolio  categories  and  assigning  each  investment  to 
a  portfolio  category;  (2)  ensuring  that  investments  have  been  analyzed  and 
their  cost,  benefit,  schedule,  and  risk  data  validated;  and  (3)  examining  the 
mix  of  investments  across  the  portfolio  categories  in  making  funding 
decisions. 


INS  does  not  assess  all  its  IT  projects  in  making  selections  for  funding. 
While  INS  has  defined  common  portfolio  categories,  it  is  not  using  them  to 
manage  its  investments.  INS  has  created  eight  portfolio  categories  and 
assigned  all  of  its  investments  to  one  of  the  portfolios.  However,  the  IRB 
has  not  analyzed  these  investments,  including  both  proposed  and  ongoing 
projects,  based  on  validated  cost,  benefit,  schedule,  and  risk  data.  Without 
these  meaningful  data,  the  IRB  cannot  compare  its  investments  across 
portfolio  categories.  As  a  result,  the  IRB  caimot  make  trade-offs  between 
investment  alternatives,  determine  which  projects  contribute  most  to 
agency  performance,  or  eliminate  redimdant  systems.  Table  10  summarizes 
the  ratings  for  each  key  practice  and  the  specific  findings  supporting  the 
ratings. 
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INS  Is  Not  Managing  Its  IT  Investments  as  a 

Complete  Portfolio 

Table  10:  Summary  of  Ratings  and  Evidence  for  the  Portfolio  Development  Critical  Process 

Key  practice 

Rating 

Summary  of  evidence 

Organizational 

commitment 

1 .  The  organization  has  written 
policies  and  procedures  for 
establishing  and  maintaining  the 
portfolio  development  process. 

Not  executed 

INS  does  not  have  written  policies  or  procedures  for 
establishing  and  maintaining  the  portfolio  development 
process. 

Prerequisites 

1 .  Adequate  resources  are  provided 
for  executing  the  portfolio 
development  process. 

Executed 

According  to  INS,  it  has  adequate  resources  available  to 
execute  the  portfolio  development  process.  These 
resources  Include  the  ESC  members. 

2.  Board  members  exhibit  core 
competencies  in  portfolio 
development. 

Not  executed 

The  IRB/ESC  members  do  not  collectively  analyze  all  IT 
Investments  using  portfolio  selection  criteria  and  thus 
have  not  exhibited  core  competencies  In  portfolio 
development. 

3.  Individual  IT  investments  have 
been  analyzed  and  their  CBSR  data 
have  been  validated. 

Not  executed 

INS  indicated  in  Its  self-assessment  that  this  key  practice 
was  “not  executed.” 

4.  The  organization  has  defined  its 
common  portfolio  categories. 

Executed 

INS  has  defined  the  following  eight  portfolio  categories. 
They  are  Enforcement,  Inspections,  Examinations, 
Corporate,  Management,  Infrastructure,  Biometrics,  and 
IRM  Operations. 

Activities 

1 .  Each  IT  investment  board  assigns 
investment  proposals  to  a  portfolio 
category. 

Executed 

Each  IT  project  is  assigned  to  a  portfolio  category. 

2.  Each  IT  investment  board 
examines  the  mix  of  proposals  and 
investments  across  the  common 
portfolio  categories  and  makes 
selections  for  funding. 

Not  executed 

INS  does  not  examine  the  mix  of  investments  across  all 
portfolio  categories. 

3.  Each  IT  investment  board 
approves  or  modifies  the  annual 
CBSR  expectations  for  each  of  Its 
selected  IT  investments. 

Not  executed 

INS  indicated  in  its  self-assessment  that  this  key  practice 
was  “not  executed.” 

4.  A  repository  of  portfolio 
development  information  is 
established,  updated,  and 
maintained. 

Executed 

A  repository  of  portfolio  information  has  been  created 
and  is  being  maintained. 
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INS  Is  Not  Managing  Its  IT  Investments  as  a 
Complete  Portfolio 


INS  Does  Not  Oversee 
IT  Investments'  Cost, 
Benefit,  Schedule,  and 
Risk  Performance 


The  purpose  of  the  portfolio  performance  oversight  critical  process  is  to 
ensure  that  each  IT  investment  achieves  its  cost,  benefit,  schedule,  and  risk 
expectations.  This  critical  process  builds  upon  the  IT  Project  Oversight 
critical  process  by  adding  the  elements  of  benefit  measmement  and  risk 
management  to  an  organization's  investment  control  capacity.  Executive- 
level  oversight  of  project-level  risk  and  benefit  management  activities 
provides  the  organization  with  increased  assurance  that  each  investment 
will  achieve  the  desired  cost,  benefit,  schedule,  and  risk  results. 


According  to  ITIM,  effective  portfolio  performance  oversight  requires, 
among  other  things,  that  the  IRB  (1)  have  access  to  up-to-date  cost,  benefit, 
schedule,  and  risk  data;  (2)  monitor  the  performance  of  each  investment  in 
its  portfolio  by  comparing  actual  project-level  cost,  benefit,  schedule,  and 
risk  data  to  the  predefined  expectations  for  the  project;  and  (3)  correct 
poorly  performing  projects. 

INS  does  not  monitor  its  investments'  performance  to  ensure  that  they  are 
meeting  cost,  benefit,  schedule,  and  risk  performance  expectations.  As 
mentioned  previously,  up-to-date  cost,  benefit,  schedule,  and  risk  data  are 
not  available.  Without  these  data,  the  ERB  is  unable  to  monitor  the 
performance  of  its  investments  to  ensure  that  they  are  achieving  their  cost, 
benefit,  schedule,  and  risk  expectations  and  to  act  when  performance 
problems  arise.  Table  11  summarizes  the  ratings  for  each  key  practice  and 
the  specific  findings  supporting  the  ratings. 
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INS  Is  Not  Managing  Its  IT  Investments  as  a 
Complete  Portfolio 


Table  1 1 :  Summary  of  Ratings  for  the  Portfolio  Performance  Oversight  Critical  Process 


Key  practice 

Rating 

Summary  of  evidence 

Organizational 

commitment 

1 .  The  organization  has  written 
policies  and  procedures  for 
monitoring  and  controlling  portfolio 
performance. 

Not  executed 

INS  has  no  policies  and  procedures  for  monitoring  and 
controlling  portfolio  performance. 

Prerequisites 

1 .  Adequate  resources  are  provided 
for  monitoring  and  controlling  the 
portfolio's  performance. 

Not  executed 

INS  indicated  in  its  self-assessment  that  this  key  practice 
was  “not  executed.” 

2.  Annual  CBSR  expectations  are 
agreed  upon  for  each  IT  investment. 

Not  executed 

INS  indicated  in  its  self-assessment  that  this  key  practice 
was  “not  executed.” 

3.  The  IT  investment  board  has 
access  to  up-to-date  actual  and 
expected  CBSR  data  in  the 
repository. 

Not  executed 

INS  indicated  in  its  self-assessment  that  this  key  practice 
was  “not  executed.” 

Activities 

1 .  Each  IT  Investment  board  monitors 
the  performance  of  each  investment 
in  its  portfolio  by  comparing  actual 
CBSR  data  to  expectations. 

Not  executed 

INS  indicated  in  Its  self-assessment  that  this  key  practice 
was  “not  executed.” 

2.  Using  established  criteria,  the  IT 
investment  board  identifies  its 
investments  that  have  not  met 
predetermined  CBSR  performance 
expectations. 

Not  executed 

INS  indicated  in  its  self-assessment  that  this  key  practice 
was  “not  executed.” 

3.  The  IT  investment  board  and  the 
project  manager  determine  the  root 
cause  of  the  poor  performance. 

Not  executed 

INS  indicated  in  its  self-assessment  that  this  key  practice 
was  “not  executed.” 

4.  The  IT  investment  board  and  the 
project  manager  develop  an  action 
plan  designed  to  remedy  the 
identified  cause(s)  of  poor 
performance. 

Not  executed 

INS  indicated  in  its  self-assessment  that  this  key  practice 
was  “not  executed.” 

5.  Corrective  actions  are  initiated  and 
outcomes  are  tracked. 

Not  executed 

INS  indicated  in  its  self-assessment  that  this  key  practice 
was  “not  executed.” 
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The  Department  of  Justice  Is  Not  Guiding  and 
Overseeing  INS'  Investment  Management 
Approach 


The  Clinger-Cohen  Act  of  1996  imposed  rigor  and  structure  on  how 
agencies  approach  the  selection  and  management  of  IT  projects.  *  Among 
other  things,  it  requires  the  head  of  each  agency  to  implement  a  process  for 
maximizing  the  value  of  the  agency's  IT  investments  and  assess  and 
manage  the  risks  of  its  IT  investments.  It  also  requires  that  the  agency  CIO 
work  with  the  agency  head  in  implementing  this  process.  As  such,  Justice  is 
responsible  for  ensuring  that  its  bureaus  and  components,  including  INS, 
implement  an  effective  IT  investment  management  process. 

Justice  has  not  provided  INS,  or  any  other  Justice  component,  sufficient 
direction,  guidance,  and  oversight  of  IT  investment  management  activities. 
While  Justice  issued  guidance  in  January  2000  describing  its  high-level 
investment  management  process,  the  guidance  does  not  address  the  need 
or  requirement  for  Justice's  components  to  implement  an  IT  investment 
management  process.  Specifically,  this  guidance  does  not  instruct  the 
components  to  establish  IT  investment  management  processes  nor  does  it 
establish  expectations  for  doing  so.  According  to  Justice  officials.  Justice 
had  not  established  these  processes  because  of  other  competing 
department  priorities,  even  though  the  department  and  its  components 
spent  about  $3  billion  on  IT  in  fiscal  years  1999  and  2000. 

Diuing  the  course  of  our  work.  Justice  began  drafting  IT  investment 
management  policy  and  guidance  documents  in  coUaboration  with  an 
intercomponent  working  group.  The  draft  policy  directs  Justice 
components  to  establish  and  use  an  IT  investment  management  process 
and  directs  the  Justice  CIO  to  monitor  the  components'  investment 
management  processes  through  periodic  briefings.  A  supplemental 
guidance  document  provides  procedures  for  developing  an  investment 
management  process.  Justice  officials  stated  that  they  plan  to  issue  the 
final  policy  by  the  end  of  December  2000  and  the  guidance  by  March  2001. 
Until  Justice  issues  its  policy  and  guidance  and  begins  monitoring  its 
components'  progress,  it  has  no  assurance  that  it  has  the  necessary 
investment  management  processes  in  place  to  maximize  the  value  of  its  IT 
investments  and  manage  the  risks  associated  with  them. 


'The  fiscal  year  1997  Omnibus  Consolidated  Appropriations  Act,  P.  L.  104-208,  renamed  both 
Division  D  (the  Federal  Acquisition  Reform  Act)  and  E  (the  Information  TechnolOjy 
Management  Reform  Act)  of  the  1996  DOD  Authorization  Act,  P.  L.  104-106,  as  the  Clinger- 
Cohen  Act  of  1996. 
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Conclusions,  Recommendations,  and  Agency 
Comments 


IT  is  critical  to  INS'  ability  to  provide  vital  services,  such  as  granting 
naturalization  benefits  and  detecting  and  preventing  the  illegal  entry  of 
aliens  into  the  United  States.  Effectively  and  efficiently  managing  IT 
requires,  among  other  things,  a  structured  approach  for  minimizing  the  risk 
and  maximizing  the  return  on  IT  investments.  However,  INS  executives  are 
making  investment  decisions  involving  hundreds  of  milhons  of  dollars 
without  vital  data  about  these  investments'  relative  costs,  benefits,  and 
risks.  As  a  result,  INS  cannot  adequately  know  whether  it  is  making  the 
right  investment  decisions,  whether  it  has  selected  the  mix  of  investments 
that  best  meets  its  overall  mission  and  business  priorities,  or  whether  these 
investments  are  living  up  to  expectations. 

INS  has  initiated  efforts  to  establish  an  IT  investment  management 
foundation.  However,  it  is  lacking  many  important  foundational  investment 
management  capabilities,  particularly  those  relating  to  controlling  projects 
against  predetermined  expectations  and  addressing  variances.  As  a  result, 
it  runs  the  serious  risk  that  its  IT  projects  will  be  late,  cost  more  than 
expected,  and  not  perform  as  intended. 

INS'  use  of  portfolio  categories  and  portfolio  managers  provides  some 
structure  to  its  portfolio  development  process  and  provides  each  business 
area  the  opportunity  to  identify  the  projects  that  it  determines  to  be  the 
most  important  to  its  performance.  However,  INS'  lack  of  performance  data 
fi’om  ongoing  projects  handicaps  the  IRB's  ability  to  perform  its  portfolio 
oversight  function.  In  addition,  the  absence  of  any  project-to-project 
comparison  limits  the  IRB's  ability  to  judge  whether  its  mix  of  investments 
best  meets  its  mission  needs  and  priorities.  As  a  result,  INS  can  have  little 
confidence  that  its  chosen  mix  of  IT  investments  best  meets  mission  goals 
and  priorities  and  that  these  investments  will  be  developed  within  an 
acceptable  level  of  risk,  on  time,  and  within  budget. 

Further,  Justice  has  a  statutory  role  under  the  Clinger-Cohen  Act  to  ensure 
that  its  component  agencies,  including  INS,  have  effective  investment 
management  processes.  Until  Justice  fulfills  this  role,  it  has  little  assurance 
that  INS,  or  its  other  components,  are  investing  the  department's  limited  IT 
resources  to  maximize  return  on  investment,  minimize  risk,  and  best 
support  mission  needs. 


Recommendations  for 
Executive  Action 


To  strengthen  INS'  investment  management  capability  and  address  the 
weaknesses  discussed  in  this  report,  we  recoimnend  that  you  direct  the 
Commissioner  of  the  Immigration  and  Naturalization  Service  to  designate 
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development  and  implementation  of  effective  IT  investment  management 
processes  as  an  agencywide  priority  and  manage  it  as  such.  Specifically, 
you  should  direct  the  Commissioner  to  do  the  following; 

•  Develop  a  plan,  within  9  months,  for  implementing  IT  investment 
management  process  improvements  that  is  based  on  stages  two  and 
three  critical  processes  and  specifies  measmable  goals  and  time  frames, 
ranks  initiatives,  defines  a  management  structure  for  directing  and 
controlling  the  improvements,  establishes  review  milestones,  and 
recognizes  any  direction  and  guidance  that  Justice  issues.  This  plan 
should  first  focus  on  those  critical  processes  in  stage  two  of  ITIM 
because,  coUectively,  they  provide  the  foundation  for  building  a  mature 
IT  investment  management  process. 

•  Submit  the  plan  to  the  Justice  CIO  for  review  and  approval. 

•  Implement  the  approved  plan  and  report  to  the  Justice  CIO,  according  to 
established  review  milestones,  on  progress  made  against  the  plan's 
goals  and  time  frames. 

F\uther,  because  the  absence  of  effective  investment  management 
processes  and  an  enterprise  architecture*  severely  limits  INS'  ability  to 
effectively  manage  its  IT  investments,  we  recommend  that  imtil  INS 
develops  a  complete  enterprise  architecture  and  implements  the  key 
practices  associated  with  stages  two  and  three  critical  processes,  as 
described  in  this  report,  you  direct  the  Commissioner  to  limit  requests  for 
future  appropriations  for  IT  only  to  efforts  that 

•  support  ongoing  operations  and  maintenance,  but  not  mqjor 
enhancements,  of  existing  systems; 

•  support  INS  efforts  to  develop  and  implement  IT  investment 
management  processes  and  an  enterprise  architecture; 

•  are  small,  represent  low  technical  risk,  and  can  be  delivered  in  a 
relatively  short  period  of  time;  or 

•  are  congressionally  mandated. 

Further,  to  improve  Justice's  guidance  and  oversight  of  components'  IT 
investment  management  process  activities,  we  also  recommend  that  you 
direct  the  Justice  CIO  to  follow  through  on  the  department's  plans  to  issue 


*  Information  Technology:  INS  Needs  to  Better  Manage  the  Development  of  Its  Enterprise 
Architecture  (GAO/AIMD-00-212,  August  1,  2000). 
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an  IT  investment  management  policy  and  guidance  to  the  components  and 
to  ensure  that  the  policy  and  guidance: 

•  Directs  Justice  components  and  bureaus,  including  INS,  to  develop  and 
implement  IT  investment  management  processes. 

•  Instructs  Justice  components  and  bureaus  on  how  to  develop  an 
investment  management  process.  This  guidance  should  be  based  on  the 
investment  management  guidance  contained  in  this  report  and,  at  a 
minimum,  should  include  component  roles,  responsibilities,  authorities, 
and  policies  and  procedures  for  developing  an  IT  investment 
management  process. 

•  Directs  the  Justice  CIO  to  monitor  the  components'  progress  in 
developing  and  establishing  an  IT  investment  management  process  and 
take  appropriate  action  if  they  are  not  progressing  sufficiently. 


Agency  Comments  and 
Our  Evaluation 


In  written  comments  on  a  draft  of  this  report.  Justice's  Assistant  Attorney 
General  for  Administration  generally  agreed  with  our  recommendations, 
although  he  offered  minor  wording  modifications  on  two  recommendations 
that  he  said  would  increase  Justice's  ability  to  fully  implement  them.  The 
Assistant  Attorney  General  for  Administration  also  disagreed  with  om 
finding  that  Justice  is  not  guiding  and  directing  INS'  investment 
management  approach. 


Justice  generally  agreed  with  our  recommendation  that  INS  develop  and 
submit  to  Justice  a  plan  for  implementing  investment  management  process 
improvements.  However,  Justice  suggested  that  the  time  frame  for 
developing  the  plan  be  clarified  such  that  INS  has  6  months  to  develop  and 
submit  its  plan  to  Justice  once  Justice  issues  its  new  IT  investment 
management  guidance.  Because  our  recommendation  directed  INS  to 
consider  any  Justice  guidance  and  direction  in  developing  its  investment 
management  process  improvement  plan,  we  modified  the  recommendation 
to  include  an  additional  3  months  to  allow  time  for  Justice  to  issue  its 
guidance,  which  it  plans  to  do  in  March  2001. 


Justice  also  concurred  with  our  recommendation  that  INS  limit  future 
appropriation  requests  for  IT  to  certain  investment  categories  because  it 
lacks  an  enterprise  architecture  and  effective  investment  management 
processes,  but  suggested  that  we  specify  that  this  recommendation  is  in 
effect  initil  INS  completes  its  architecture  and  implements  investment 
management  processes.  Because  this  is  the  intent  of  our  recommendation, 
we  clarified  the  reconunendation  to  make  this  explicit. 
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Also  in  its  comments,  Justice  agreed  that,  while  INS  has  some  important 
investment  management  capabilities,  INS  still  needs  to  develop  effective 
investment  management  processes.  Further,  Justice  agreed  with  our 
recommendation  for  Justice  to  issue  an  investment  management  policy  and 
guidance  to  its  components,  including  INS,  that  (1)  directs  components  to 
develop  and  implement  IT  investment  management  processes,  (2)  instructs 
components  on  how  to  develop  and  implement  these  processes  based  on 
the  investment  management  framework  in  our  report,  and  (3)  ensures  that 
components'  progress  in  doing  so  is  monitored.  Moreover,  Justice  stated, 
which  we  note  in  oiu*  report,  that  it  is  now  working  with  its  components  to 
develop  an  IT  investment  management  policy  and  process,  and  it  has  made 
this  a  department  priority  for  this  year. 

However,  Justice  stated  that  oiu"  draft  report  fails  to  recognize  the  extent  of 
Justice's  oversight  of  INS'  IT  investment  management  process.  Further,  it 
disagreed  with  our  finding  that  Justice  is  not  guiding  and  directing  INS' 
investment  management  approach.  Justice  stated  that  it  has  established 
guidance  for  all  aspects  of  IT  management  that  its  components  are 
expected  to  follow  and  has  a  process  for  overseeing  components' 
management  of  their  investments.  Justice  cited  six  examples  to  illustrate 
its  point,  such  as  Justice  approval  authority  of  all  component  IT 
investments  with  life-cycle  cost  over  $1  million.  Justice  establishment  of  an 
IT  investment  board.  Justice  meetings  with  components,  including 
Attorney  General  meetings  with  the  INS  Commissioner,  and  Justice 
forwarding  of  0MB  budget  requirements  to  components. 

We  do  not  agree  with  Justice's  position.  While  we  concur  that  the  examples 
cited  by  Justice  represent  important  IT  management  functions  to  be 
performed  in  providing  management  oversight  of  individual  IT 
investments,  such  management  oversight  is  not  the  focus  of  our  findings, 
conclusions,  and  recommendations.  Rather,  our  report  addresses  Justice's 
efforts  to  ensure  that  its  components,  including  INS,  have  each  defined  and 
implemented  effective  IT  investment  management  processes.  As  such,  we 
sought  evidence  from  Justice  demonstrating  that  it  has  directed  its 
components  to  establish  such  processes,  provided  guidance  to  its 
components  on  how  to  develop  and  implement  the  se  processes,  and 
monitored  its  components'  progress  to  determine  whether  they  are 
implementing  such  processes.  However,  besides  the  steps  that  Justice 
initiated  during  the  course  of  om  inquiries  and  plans  to  take,  which  we 
have  described  in  this  report,  we  found  no  such  evidence.  Moreover, 
Justice  stated  in  its  written  comments  that  it  agreed  with  our 
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recommendation  for  it  to  provide  investment  management  process 
direction,  guidance,  and  oversight  to  its  components. 

Justice's  written  comments  and  our  evaluation  of  them  are  presented  in 
appendix  I. 
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Note:  GAO  comments 
supplementing  those  in 
the  report  text  appear  at 
the  end  of  this  appendix. 


See  comment  1 . 


US.  Department  of  Justice 


mshingion,  D  C  2(^30 


NOV  1 6  2000 


Mr.  Randolph  C.  Hite 
Director 

Information  Technology  Issues 
U.S.  General  Accounting  Office 
441  G  Street,  NW 
Washington,  DC  20548 

Dear  Mr.  Hite; 

On  October  25,  2000  you  submitted  the  General  Accounting  Office 
(GAO)  draft  report  entitled  Information  Technology:  INS  Needs  to 
Strengthen  Its  Investment  Management  Capability''  to  the 
Department  of  Justice  (DOJ)  with  a  request  for  its  review  and 
comment.  While  we  are  pleased  that  the  GAO  has  recognized  that 
the  Immigration  and  Naturalization  Service  (INS)  has  developed 
some  important  capabilities  that  are  the  basis  for  establishing 
an  effective  information  technology  investment  management  (ITIM) 
process,  we  are  concerned  that  the  GAO  failed  to  recognize  the 
extent  of  DOJ  oversight  of  the  INS  ITIM  process.  We  do  not  agree 
with  the  GAO' s  finding  that  the  DOJ  is  not  adequately  guiding  and 
overseeing  INS'  investment  management  approaches. 

The  Department  has  an  established  system  of  guidance  for  all 
aspects  of  information  technology  (IT)  management  that  its 
components  are  expected  to  follow  and  has  a  process  for 
overseeing  components'  management  of  their  investments. 
Specifically: 

•  DOJ  policy  requires  components  to  obtain  approval  from  the 
Department's  Chief  Information  Officer  (CIO)  for  all  IT 
acquisitions  with  a  life-cycle  cost  over  $1  million.  This 
requirement  is  contained  in  DOJ  Order  2830. ID,  with  the 
revised  threshold  of  $1  million  documented  in  a  memorandum 
from  the  DOJ  CIO  on  April  10,  2000. 
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Mr.  Randolph  C.  Hite  2 

•  The  DOJ  CIO  and  senior  staff  meet  with  management 
representatives  of  the  major  components  quarterly  to  review 
project  status  and  address  problem  issues  as  they  occur. 

•  The  Department  has  established  an  IT  Investment  Board 
chaired  by  the  Deputy  Attorney  General  which  addresses  major 
cross-cutting  IT  management  issues. 

•  As  part  of  the  annual  DOJ  budget  call,  DOJ  CIO  staff  inform 
components  of  specific  information  required  to  justify 
support  for  new  and  ongoing  IT  investments  and  to  meet  the 
Office  of  Management  and  Budget  Exhibit  53  and  Exhibit  300B 
requirements . 

•  In  March  2000,  DOJ  CIO  staff  developed  and  distributed  to 
all  components  an  extensive  IT  Systems  Development  Life 
Cycle  guide. 

In  addition,  the  Attorney  General  meets  regularly  with  the 
Commissioner  for  INS  to  discuss  a  range  of  INS  management  issues. 
In  particular,  the  Attorney  General  asked' the  INS  Commissioner  to 
provide  a  list  of  INS  IT  investment  priorities  which  the  INS  has 
provided.  The  Attorney  General  regularly  reviews  the  IT 
investment  priorities  with  the  Commissioner. 

Although  we  are  in  general  agreement  with  the  GAO 

recommendations,  we  believe  the  DOJ  would  be  better  able  to  fully 
implement  these  recommendations  if  the  GAO  accepts  our  suggested 
modifications.  With  respect  to  the  recommendation  that  INS 
develop  a  plan  to  implement  comprehensive  IT  investment 
management  practices  and  that  they  submit  the  plan  to  the 
Department's  CIO  for  review  and  approval,  however,  we  would 
suggest  that  the  time  frame  for  developing  the  new  INS  ITIM  plan 
be  adjusted  so  that  INS  has  six  months  to  submit  its  plan  after 
the  Department  CIO  issues  his  new  IT  investment  guidance.  This 
would  allow  the  INS  time  to  develop  a  plan  consistent  with  that 
guidance  and  thereby  meet  with  DOJ  approval. 

We  concur  with  the  GAO  recommendation  that  INS  limit  its  new  IT 
investments  until  it  has  completed  its  IT  architecture  and 
implemented  its  IT  investment  management  processes.  However,  to 
allow  the  INS  to  direct  its  near-term  funding  in  accordance  with 
the  recommendations  and  to  make  future  requests  once  it  has 
satisfied  the  requirements  of  an  advanced  capability  maturity 
model,  we  request  that  the  GAO  add  language  to  show  that  the  GAO 
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recominendation  to  the  Attorney  General  is  in  effect  only  until 
INS  has  developed  its  enterprise  architecture  plan  and  its  IT 
investment  management  process.  As  currently  written,  the 
recommendation  limits  the  INS  from  making  any  future  requests  for 
IT  enhancements  unless  they  meet  the  criteria  contained  in  the 
recommendation . 

Finally,  we  concur  with  the  GAO  recommendation  that  the  DOJ  CIO 
issue  new  policy  and  guidance  on  IT  investment  management  and 
enforce  that  policy.  The  Department  has  been  working  with 
components  to  develop  a  new  Department  IT  investment  management 
policy  and  process  and  has  made  this  a  management  priority  for 
this  year. 

We  hope  you  will  find  these  comments  beneficial  in  completing 
your  report.  If  you  have  any  questions  regarding  these  comments, 
please  contact  Vickie  L.  Sloan,  Director,  Audit  Liaison  Office, 
Justice  Management  Division. 

Sincerely 
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The  following  are  GAO's  comments  on  the  Department  of  Justice's  letter 
dated  November  16,  2000. 


Q.^Q  CoimnentS  agree  with  Justice's  statement  that  it  has  established 

guidance  for  all  aspects  of  IT  management  that  its  components  are 
expected  to  follow  and  has  a  process  for  overseeing  components' 
management  of  their  investments.  While  we  concur  that  the  examples 
cited  by  Justice  represent  important  IT  management  functions  to  be 
performed  in  providing  management  oversight  of  individual  IT 
investments,  such  management  oversight  is  not  the  focus  of  our 
findings,  conclusions,  and  recommendations.  Rather,  our  report 
addresses  Justice's  efforts  to  ensure  that  its  components,  including 
INS,  have  each  defined  and  implemented  effective  IT  investment 
management  processes.  To  this  end,  we  sought  evidence  from  Justice 
demonstrating  that  it  has  directed  its  components  to  establish  such 
processes,  provided  guidance  to  its  components  on  how  to  develop  and 
implement  these  processes,  and  monitored  its  components'  progress  to 
determine  whether  they  are  implementing  such  processes.  Besides  the 
steps  that  Justice  initiated  during  the  course  of  our  inquiries  and  plans 
to  take,  which  we  have  described  in  our  report,  we  found  no  such 
evidence.  Moreover,  Justice  stated  in  its  written  comments  that  it 
agrees  with  our  recommendation  for  it  to  provide  investment 
management  process  direction,  guidance,  and  oversight  to  its 
components. 

2.  Because  oiu  recommendation  directed  INS  to  consider  any  Justice 
guidance  and  direction  in  developing  its  investment  management 
process  improvement  plan,  we  have  modified  our  recommendation  to 
incorporate  Justice's  suggestion  that  INS  have  6  months  to  develop  and 
submit  its  plan  to  Justice  after  Justice  issues  its  new  IT  investment 
management  guidance. 

3.  It  was  our  intent  that  INS  limit  its  future  appropriation  requests  for  IT 
to  certain  investment  categories  only  until  it  completes  its  architecture 
and  implements  investment  management  processes.  As  a  result,  we 
have  clarified  the  recommendation  to  make  this  explicit. 
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